rd 3 Asia-Pacific Conference on Welcome from the Systems Engineering General Chair

APCOSE 2009 Singapore Welcome to the 19th Annual INCOSE International Symposium (INCOSE 2009), which will be held in conjunction with the 3rd July 20 - 23 Asia-Pacific Conference on Systems Engineering (APCOSE 2009) from 20-23 July 2009. As the General Chair, I have the pleasure to welcome you to the choice international forum of the systems engineering community. The theme of the symposium, East meets West: The Human Dimension to Systems Engineering, highlights the human cognitive dimension as an integral part of systems thinking and systems engineering processes across different cultures. Moreover, human capital and capability form the fundamental basis for large-scale systems engineering.

This is the first time that the INCOSE Symposium is held in Singapore and in Asia. Jointly hosted by INCOSE Region VI Chapters of Australia, Beijing, Japan, Korea, Singapore and Taiwan, the symposium reflects the growth of systems engineering as an emerging discipline in the region. This premier event will present researchers, students, educators, academics, professionals, public and private organisations from the East and East Meets West the West the opportunity to share and deliberate on the latest The Human Dimension to Systems Engineering systems engineering developments in education, research and applications. I believe this mutual exchange of ideas across different cultures can only spur greater dynamism and thinking in stretching the existing boundaries of systems engineering.

Hosted by the Region VI Chapters of Finally, Singapore with its strategic location and world-class Australia, Beijing, Japan, Korea, infrastructure, serves as the logical meeting place for industry, Singapore and Taiwan academia and government from East and West to network, develop and grow partnerships. I strongly encourage you to attend INCOSE 2009 which, I am certain, will be a fulfilling and rewarding experience.

I look forward to have you join us in Singapore for INCOSE Host Technical 2009. Committee Programme YEO Tat Soon - INCOSE 2009 General Chair APCOSE 2009 Host Committee

General Chair and Co-chairs Group Ceremony & Plenary Group YEO Tat Soon, Singapore LEO Tin Boon, Singapore COOK Steven, Australia CHUA Siew Ting Pearly, Singapore WANG Fei Yue, China OH Pei Tze Regine, Singapore SHENG Zhaohan, China Xuan-Linh TRAN, Australia KOO Mum Thiam, Singapore Main Co-ordinator / Secretariat SIAK Weng Wah, Singapore Sponsorship & Exhibition Group YEOH Lean Weng, Singapore Technical Group CHUA, Anthony, Singapore POH Kim Leng, Singapore HUYNH Thomas, USA Marketing & Website Group NG Kien Ming, Singapore LEE Yeaw Lip Alex, Singapore OHKAMI Yoshiaki, Japan CHIA Eng Seng Aaron, Singapore JIANG Zhibin, China SIAK Weng Wah, Singapore YANG Weiyuan, China CHUA Siew Ting Pearly, Singapore CHNG York Wee Janey, Singapore Academic Forum Group Xuan-Linh TRAN, Australia KASSER Joseph, Singapore PENG Yuan Shi Willy, Taiwan Logistics, Entertainment and Social Events Group LEE Tsu Tian, Taiwan SIAK Weng Wah, Singapore WONG Hsiao-Szu, Singapore SIM Kah Eng Queenie, Singapore

Technical Programme APCOSE 2009 Technical Programme

Session 1 Monday 20 July Specific Topics 1 Session 2 Monday 20 July Specific Topics 2 Session 3 Tuesday 21 July Specific Topics 3 Session 4 Tuesday 21 July Design Session 5 Tuesday 21 July Systems Engineering Session 6 Wednesday 22 July Specific Topics 4 Session 7 Wednesday 22 July SE Process Session 8 Wednesday 22 July Specific Topics 5 Session 9 Thursday 23 July Specific Topics 6 Session 10 Thursday 23 July Specific Topics 7 APCOSE 2009 Session 1 – Monday 20 July

Track Specific Topics 1 Session Chair Gary LANGFORD

AP 01-1 Predicting Outbreak of Economic Malaise in Financial Networks Thomas V. Huynh, John S. Osmundson, The Naval Postgraduate School

AP 01-2 Advising Policy Makers – A Bayesian Approach Michael A. Duffy, National Renewable Energy Laboratory

AP 01-3 A Practical Application of MBSE - The Automated Parking System Philip Simpkins, Adam Kleinholz, Joe Maley, Vitech Corporation

Technical Programme APCOSE 2009 Session 2 – Monday 20 July

Track Specific Topics 2 Session Chair Aaron CHIA

AP 02-1 Multi-Culture Structure of Opera, “Back to the Origin” based on SE Process Sung Ki Min, Institute of Systems Engineering, Inc; Chan Hae Lee, College of Music, Yonsei Univ

AP 02-2 Creating and Valuing Flexibility in Systems Architeching: Transforming Uncertainties Into Opportunities Using Real Options Analysis Ho Wei Ling Angela, Ng Chu Ngah, Defence Science and Technology Agency; de Neufville Richard, Massachusetts Institute of Technology

AP 02-3 A Real Options Framework for Defence R&D Investments in Capability Development Choon Keat Ang, Defence Science and Technology Agency; Kah Hin Chai, National University of Singapore

Technical Programme APCOSE 2009 Session 3 – Tuesday 21 July

Track Specific Topics 3 Session Chair LIM Horng Leong

AP 03-1 Applying Systems Thinking to Solving the Problem of Vertically Integrating Taiwan’s Small and Medium Sized Enterprises Joseph E. Kasser, National University of Singapore; Willy Y.S. Peng, Overseas Chinese Institute of Technology

AP 03-2 Systems Engineering in the Conceptual Stage: A Teaching Case Study Zhao Yang Yang, Joseph E. Kasser, National University of Singapore; Derek Hitchins, ASTEM Consultant Systems Architect

AP 03-3 A Systematic Approach Of Quality Verification For Efficient Replacement In Case Of Component EOL Takayuki Tomaru, Hidekazu Nishimura, Masaru Nakano, Keio University; Kosuke Ishii, Stanford University

AP 03-4 Model for Estimating Logistics and Operations Support Costs in Command Control and Communications Projects Choon-Bong Wong, Yeow-Koon Tay, Tiong-Lee Ng, Defence Science and Technology Agency

Technical Programme APCOSE 2009 Session 4 – Tuesday 21 July

Track Design Session Chair WANG Tsung Jung

AP 04-1 Systems Engineering Through The Human Lenses Lean Weng Yeoh, Horng Leong Lim, Seng Guan Soh, Defence Science and Technology Agency

AP 04-2 Multi-objective Optimization of Two Functions with Three Variables and its Application Masataka Urago, Hidekazu Nishimura, Yoshiaki Ohkami, Keio University

AP 04-3 Case Study of the Military Aircraft R&D Programs for Systems Engineering Application Nam Kyung Ko, Yong Soo Kwon, Korea National Defense University; Chul Whan Kim, Korea Council on Systems Engineering; Sung Eun Lee, Republic of Korea Air Force

Technical Programme APCOSE 2009 Session 5 – Tuesday 21 July

Track Systems Engineering Session Chair Thomas HUYNH

AP 05-1 Tracking the Technical Critical Path: Systems Engineering Meets Project Management James R. Armstrong, Stevens Institute of Technology

AP 05-2 Becoming a Systems Engineer Roy de Souza, Gary Langford, Naval Postgraduate School

AP 05-3 Reengineering Systems Engineering Joseph Kasser, National University of Singapore; Derek Hitchins, ASTEM, Consultant Systems Architect; Thomas V. Huynh, Naval Postgraduate School

Technical Programme APCOSE 2009 Session 6 – Wednesday 22 July

Track Specific Topics 4 Session Chair TONG Chee Kong

AP 06-1 PLC Home Network System for Earthquake Disaster Prevention Daisuke Honda, Kyosuke Harayama, Masahiro Inoue, Shibaura Institute of Technology

AP 06-2 System Integration of Motorcycle Driving Stability Control Using SysML Shaopeng Zhu, Hidekazu Nishimura, Laurent Balmelli, Keio University

AP 06-3 Towards Better Adaptation of Systems Engineering To Japanese Space Program Masashi Okada, Japan Aerospace Exploration Agency, Keio University; Masaru Nakano, Keio University

AP 06-4 Study on Interface Management based on Model Sung Gyun Oh, Young Won Park, Chan Mook Kim, Graduate School of Ajou University

Technical Programme APCOSE 2009 Session 7 – Wednesday 22 July

Track SE Process Session Chair Hidekazu NISHIMURA

AP 07-1 On Systems Engineering Application Model for Korean National R&D Projects in Railway Systems Domain Yo-Chul Choi, SEKOREA; Jae-Chon Lee, AJOU University

AP 07-2 A Systems Approach for Managing Concurrent Product Development Projects Jun Lin, Xi’an Jiaotong University; Tsan Sheng Adam Ng, National University of Singapore

AP 07-3 Systems Engineering’s Role in Today’s Mega Projects - “The Systems Integrator” Leon “Lee” J. Pandazides, LJP & Associates

Technical Programme APCOSE 2009 Session 8 – Wednesday 22 July

Track Specific Topics 5 Session Chair CHAN Weng Tat

AP 08-1 Using Systems Engineering to Implement Corporate Social Responsibility to Meet the Challenges of a Globalized World Annik Magerholm Fet, Norwegian University of Science and Technology

AP 08-2 DSTA’s Journey in Systems Architecting Tan Yang How, Teo Siow Hiang, Lee Keen Sing, Lim Hang Sheng, Defence Science and Technology Agency

AP 08-3 Emerging Behaviors in Large Scale Systems: The Singapore Education System Ori Sasson, Singapore Management University

Technical Programme APCOSE 2009 Session 9 – Thursday 23 July

Track Specific Topics 6 Session Chair Xuan-Linh TRAN

AP 09-1 The Integrated Design Environment of Luna Probe-Based Software Product Line Hu Zhixin, Lin Yinming, China Academy of Space Technology

AP 09-2 Development of Aerospace System Engineer Technology Zheng Jinjun, Lin Yiming, Chu Haibin, China Academy of Space Technology

Technical Programme APCOSE 2009 Session 10 – Thursday 23 July

Track Specific Topics 7 Session Chair NG Kien Ming

AP 10-1 Development Schemes of Spacecraft Systems Engineering Techniques Yu Houman, Hao Wenyu, Yuan Jungang, Lin Yiming, Beijing Institute of Space System Engineering

AP 10-2 The Integrated System for Spacecraft Simulation and Test Sun Ya-Nan, Tu Xin-Ying, Xiang Kai-Heng, Li Zhi, China Academy of Space Technology

Technical Programme Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 13

Predicting Outbreak of Economic Malaise in Financial Networks

Thomas V. Huynh John S. Osmundson Department of Systems Engineering Department of Information Sciences The Naval Postgraduate School The Naval Postgraduate School Monterey, CA 93943, USA Monterey, CA 93943, USA [email protected] [email protected]

Copyright © 2009 by Thomas V. Huynh and John S. Osmundson. Published and used by APCOSE with permission.

Abstract. More than a year ago, the financial storm struck the United States. Now, the U.S. is in a financial crisis ─ an outbreak of failed financial institutions, brought about by the spreading of the financial malaise, i.e., illiquid collateralized debt obligations (CDOs), through the financial system. But no analytical work has been found that would predict the occurrence of the outbreak or explain the magnitude of the collapse of the financial institutions ─ an emergent behavior in an unregulated financial system of systems. To predict such a meltdown of the financial system, in this paper, we adapt a theoretical framework, developed by Newman in his work on the spread of epidemic disease on networks, to the propagation of the financial malaise. Specifically, we represent the financial system of systems as a graph or a network, calculate the financial malaise transmissibility as a function of the mean spreading time, and compute the average number of failed financial institutions as a function of the financial malaise transmissibility. Together, the financial malaise transmissibility and the average number of failed financial institutions indicate the observed behavior of the deregulated CDO market. The analytical approach in this paper forms an analytical basis for predicting consequences of deregulation and the causes of this financial meltdown. This approach has been also applied to the prediction of emergent behavior of the U.S. electric power grids under deregulation. Introduction Three pillars of systems engineering are systems engineering management, systems engineering methodology, and systems engineering methods and tools [Sage, 1992]. Systems engineering methodology involves (i) system definition, (ii) system design and development, and (iii) system deployment. Systems management consists of a task management structure, managing systems engineering tasks, and decision making with respect to system development. The third pillar, systems engineering methods and tools, is needed to support systems engineering management and systems engineering methodology [Huynh, 2008]. This paper deals with systems engineering methods and tools. Specifically, it deals with an application of a graph (network) method to analyze emergent behavior of the financial system of systems. This graph method has also been applied to ascertain the emergent behavior of the North America electrical power grids under deregulation [Huynh and Osmundson, 2008]. The discussion of the recent fallout of the US financial system benefits from the material in [Atlas and Dreier, 2007; Economist, 2008; Wikipedia, 2008; Mysmp, 2008; Roudini, 2008; Credit Crisis, 2008; Isidore, 2008]. More than a year ago, the financial storm struck the United States (US). To date, 17 banks, including five giant investment banks, have thus far disappeared in a short time.

The money market funds, security dealers, hedge funds, and the other non-bank financial institutions that defined deregulated American finance are in trouble. Just in two turbulent weeks in October, 2008, the US government nationalized the two mortgage giants, Fannie Mae and Freddie Mac, bailed out the world’s largest insurance company, AIG, “extended government deposit insurance to $3.4 trillion in money-market funds, temporarily banned short-selling in over 900 mostly financial stocks, most dramatic of all, pledged to take up $700 billion of toxic mortgage-related assets on to its books.” [Economist, 2008] More than 100 banks nationwide are expected to fail next year [Isidore, 2008]. The root cause is known to be the “highly deregulated, lightly regulated, market-based system of allocating capital dominance by Wall Street.” [Economist, 2008] Until the late 1970s, Washington tightly regulated the savings-and-loan industry; homeownership increased and foreclosures were few then. In the early 1980s, interest-rate caps were eliminated and subprime lending became more feasible for lenders. The deregulation of banking eventually led to the disappearance of the Savings and Loans (S&Ls). They were replaced by banks, insurance companies, credit card firms, mortgage banks and brokerage firms, which, untouched by federal regulations, devised the subprime and predatory loans that put desperate home buyers at risk. Instead of holding mortgages on their books, banks pooled and sold their repackaged assets (mortgages) to investment banks, which in turn, through their own corporate structures, pooled mortgage-backed securities. These mortgage-backed securities are a most common debt instrument within collateralized debt obligations (CDOs). Of the three tranches ─ senior, mezzanine, and equity ─ in a CDO, the equity tranche, known as the "toxic waste", is most vulnerable to default. A CDO thus holds assets as collateral and sells the cash flow from the different tranches to investors, who are entitled to its interest income and principal. An investment bank creating the CDOs presents them as investments and it makes money from a commission at the time of issue and management fees during the life of the CDO. Many CDOs included derivatives backed by mortgages ─ including risky, subprime mortgages. In early 2007, the skyrocketing of defaults in the mortgage market was weakening the assets underlying CDOs. The CDO market then suffered from widening losses. The market for CDOs' underlying assets began to vanish. Suddenly it was impossible to dump the subprime-mortgage derivatives and other securities held by the CDOs. Big buyers of CDOs such as hedge funds, commercial and investment banks, and pension suddenly found themselves in trouble. By early 2008, under the weight of the CDO market collapse, much of the derivatives market plummeted. Then, in March 2008, Bear Stearns, collapsed. Eventually, the fallout spread. By the middle of 2008, banks disappeared, and financial institutions that defined deregulated American finance were in trouble. An outbreak of failed financial institutions occurred. We now briefly recapture the workings of the financial world or the financial system of systems in trouble. Refer to Fig. 1. Mortgage borrowers obtained loans from lenders, who then sold the mortgages to investment banks, which set up corporations to issue and sell CDOs to investors, who, insured by insurance companies, are entitled to the CDO’s interest income and principal. Now, when there were no more interest income and principal to pay the investors, these CDOs became illiquid. We call illiquid CDOs a ‘financial malaise’. Holding these illiquid CDOs is what we term ‘catching the malaise’. The outbreak of failed financial institutions, i.e., the spreading of the malaise through the financial system of systems, is what we want to predict. The collapse of the housing bubble was correctly predicted, not surprisingly as home prices were

2

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 13 increasing at up to 20% a year. To the authors’ knowledge, there had not been any prediction that the financial system would collapse as a result of the collapse of the housing bubble. Economic experts admitted that no one saw this coming. Certainly Alan Greenspan didn’t! The doomed economic situation was speculated and warned [Roubini, 2008]. But no rigorous, analytical work has been found that would predict the occurrence of the outbreak or explain the magnitude of the collapse of the financial institutions. The purpose of this paper is thus to show how the financial fallout spread, using the framework employed in [Huynh and Osmundson, 2008] – spread from banks to capital markets and their investors in the U.S. and abroad. To this end, we apply the theoretical framework employed in Newman’s work [Newman, 2002] on the spread of epidemic disease on networks to show the spread of the financial malaise in the financial system (a.k.a., Wall Street). The theoretical framework includes: Modelling of the financial system of systems as a bipartite graph; modelling of the financial malaise transmissibility; computing the average number of failed financial institutions, using the theoretical framework in [Newman, 2002]; and combining the financial malaise transmissibility and the average number of failed banks to obtain the sensitivity of the spread to the duration of the malaise spreading. Parenthetically, besides the graph theory, the mathematical tools used here involve probability theory and calculus, with which systems engineers should be familiar. Our goal in this paper is to demonstrate: • Emergent behavior in an unregulated financial system ─ in this case, the collapse of the financial system under that of the weight of the CDO market. • The applicability of the analytical approach espoused here to the prediction of consequences of deregulation and the causes of this financial meltdown. The rest of the paper is organized as follows. We discuss the modelling of the financial system of systems as a graph or a network. We then describe the modelling and the computation of the financial malaise transmissibility and the computation of the average number of failed banks. We next discuss the results. We end with some concluding remarks. Financial System as a Graph We start with the general network representing the financial system of systems. We then simplify it by aggregating borrowers and lenders as single nodes. We then have just a bipartite graph, which leads to a one-mode network. A simplified financial system of systems treated here consists of borrowers, lenders, investment banks with associated special-purpose entities (corporations), investors, and insurers (Fig. 1). Borrowers obtain mortgages from lenders. Lenders then sell these mortgages or loans to investment banks; the borrowers now pay their mortgages directly to the investment banks. The investment banks then set up corporations to synthesize these loans into CDOs, which are sold to interested investors. Some of these investors are also insured by some insurance companies.

Figure 1. A simplified financial system

As the mortgage borrowers now make payments directly to the investment banks, the borrowers and the lenders can be aggregated into borrowers. The corporations set up by the investment banks can be grouped with the investment banks. The system now has the structure depicted in Fig 2.

Figure 2. The reduced contact network

Whereas defaulting by borrowers on their mortgage leads to the loss of value to the CDOs, the borrowers are not part of the investment institutions. The financial system can be reduced to just a conglomerate of interacting investment banks, investors, and insurers, enclosed by the large rectangular box (Fig. 3). This contact network has the structure of a bipartite graph.

4

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 13

Figure 3. The bipartite contact graph

Figure 4. A bipartite graph model of a network of financial institutions (left) and the corresponding one-mode network of investors and insurers (right)

For simplicity and clarity, we assume the contact network to consist of three investment banks with special-purpose entities (corporations), five investors, and two insurers. This network may be considered to belong to the class of contact networks [Meyer et al., 2006], which are known to have a bipartite structure and strongly non-Poisson degree distributions [Newman, 2002]. This network can thus be represented by a bipartite graph, whose vertices are the investment banks and the investors and whose arcs run between the investment banks and the investors. The arcs

connecting the investors and the insurers represent the relationships between the investors and the insurers. The bipartite graph is then projected onto the so-called one-mode network (unipartite graph) on the right in Fig. 4. In the one-mode network, the vertices represent investors and the insurers. A financial fallout occurs in the network as an epidemic disease on a human network [Newman, 2002]. The malaise first appears at a randomly chosen vertex in the network. (As has been played out in the entire world, this malaise propagates through the network.) The effect of the malaise is the collapse of the financial institutions. The percolation the failing of the financial institutions through the network depends on both the level of the faltering of the assets underlying the CDOs and the structure of the network [Newman, 2002], [Meyers, 2007]. Excerpted from Huynh and Osmundson (2008), the structure of the network is reflected by the average degree of the network, which is obtained from a degree probability distribution. The degree of a vertex indicates the number of vertices to which it is connected. As this type of network is known to admit non-Poisson degree distributions, we assume the probability of a randomly chosen vertex having degree k in the one-mode network is of the truncated power-law form

k − p = ℵk −α e κ k , k ≥ 1 , in which α > 1, κ is the exponential cutoff frequency, and , the normalization constant, is given by 1 ℵ = −1 / κ Liα ()e

where Lin ()x is the nth polylogarithm of x . As pointed out in [Newman et al., 2001], and [Newman, 2000], the exponential cutoff is included because it appears to exist in many real-world graphs and it makes the distribution normalizable for all α. Cluster of Failed Financial Institutions

Following Newman [Newman, 2002], let rij denote a fixed probability of malaise propagation per unit time associated with the arc connecting the i th vertex and the jth vertex. It is the probability that the ith vertex, having the malaise, will transmit it to the jth vertex in a given time interval. The ith vertex is transmitting the malaise for a continuous time τ, assumed to be chosen from a distribution g(τ ) . The quantity r summarizes core aspects of malaise transmission. If r is ij ij assumed to be an independent identically distributed (iid) random variable chosen from a

distribution f ( r ), then Tij is also an iid random variable. Therefore, the spread of the malaise will depend only on the mean probability of transmission between the vertices (hence, the malaise transmissibility), which is given by [Newman, 2002]

∞∞ TT==−1()() ddrττf r g e−rτ , ij ∫∫ 00

where denotes the expectation value.

Calculation of Malaise Transmissibility. In the absence of real data, the per unit time

6

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 13 probability of mortgage defaulting is assumed to admit a uniform distribution. It then follows that the per unit time probability of investment banks running into financial trouble (malaise) is uniformly distributed. That is, rUij ~0,1[ ]. We also assume that the time τ for which a troubled investment bank results in another troubled investment bank admits a uniform distribution τ ~0,U []γ for some maximum timeγ . Both f ()r and g(τ ) are thus uniform distributions. A simple calculation then yields 1 T =−1Ein()γ γ (1) where Ein(γ ) is the Ein-function [Abramowitz and Stegun, 1964]. Fig. 2 depicts the average financial malaise transmissibility increases as a function of the mean time of the financial malaise spreading.

Figure 2. Transmissibility as a function of time Calculation of the Average Number Failed Banks. The discussion in this section follows Huynh and Osmundson (2008). We are interested in the size of the cluster of failed investment institutions. It is indicative of the effect of the financial malaise transmission (propagation) and the breakout of the financial system. For a fixed network, as in the case of the financial system of systems, the financial malaise transmission rate threshold exists; only small, finite-sized clusters are possible below the threshold, above which the breakout becomes untenably controlled. Following Newman et al. [3], we introduce the generating function for the probability distribution of the degree of a randomly chosen vertex,

∞ k Go ()x = ∑ pk x k=0 and the generating function for the probability distribution of the degree of a vertex at the end of a randomly chosen arc,

∞ k ∑ pk x G x = k=0 1() ' G0 ()1 , ' where G0 ()x is the derivative of G0 ()x with respect to x .

The generating function for the probability distribution of the occupied arcs from a randomly chosen vertex is [Newman, 2001]

Go (x;T ) = Go (1 + (x − 1)T )

Similarly, the generating function for the probability distribution of the number of occupied arcs from the vertex at the end of a randomly chosen arc is

G1 (x;T ) = G1 (1 + (x − 1)T )

Now the generating function H 1 ()x;T for the total number of failed investment institutions as a result of a single transmission along an arc in the network must satisfy a self-consistency condition of the form [Newman et. al., 2001; Moore and Newman, 2001]

H 1 (x;T ) = xG1 (H 1 (x;T );T )

The generating function H 0 ()x;T for the distribution of the number of failed investment institutions affected by a single failed investment institution is

H o (x;T ) = xG0 (H 1 (x;T );T )

The average size s of a cluster of failed investment institutions is now given by TG' (1) s = H ' 1;T = 1 + 0 0 () ' 1 − TG1 ()1 (2) For the truncated power-law network, TG' (1) s = 1+ 0 ' 1−TG1()1 2 Li e−1/ κ T = 1+ []α−1() −1/ κ −1/ κ −1/ κ Liα ()()e []Liα−1 e ()T +1 − Liα−2 ()e T (3)

As in [Newman, 2001], we consider α = 2 and the three different values of the cutoff frequency κ of 3, 5, 10, and 20. The results generated by (3) are captured in Fig. 3, which displays the graphs of the average scramble size s as a function of transmissibilityT , parameterized by κ , for the fixed value of α. Parenthetically, Fig. 3 shows the same results produced in [Newman, 2001]. The critical value of transmissibility, at which the average size s becomes unbounded, varies with the value of the cutoff frequency.

8

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 13

Figure 3. Average number of failed financial institution as a function of transmissibility Discussion of Results As shown Fig. 3, as Τ approaches a certain threshold value, the cluster of failed investment institutions becomes extremely large, the meltdown of the financial system. Below this threshold the cluster is manageable. For example, as T approaches the threshold values of 0.6 and 0.9 corresponding respectively to the cutoff frequencies of 5 and 3, the cluster of failed investment institutions becomes large.

It follows from (1) and (3) that the average number of failed banks increases with the mean spread time according to

(4)

Fig. 4 captures the result in (4). It shows the trend of the average number of failed banks growing with the mean malaise spread time γ (in months), obtained with the analytical approach for the case of α = 2 and κ = 3. As shown in Fig. 5, the data (represented by the markers) on the number of failed banks (over the months from 2005 and the early part of 2009, published by the Federal Deposit Insurance Corporation [FDIC, 2009], indicate an exponential trend depicted by the solid line. The theoretically result (4) for the case of α = 2 and κ = 3 thus reasonably approximates the growing trend of the number of failed banks during those months.

Figure 4. Average number of failed financial institution vs. the mean spread time

Figure 5. The monthly number of failed banks from 2005 to 2009. [Source: http://www.fdic.gov/bank/individual/failed/banklist.html]

In addition, as shown in Fig. 6, the average number of failed banks over a mean spread time (represented by the solid line), as obtained with the analytical approach for the case of α = 2 and κ = 3, appears to agree with the number of failed banks averaged over the number of months (represented by the markers) obtained with the FDIC data.

10

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 13

Figure 6. Agreement between predicted results and actual data Conclusion In this work we apply Newman’s work [Newman, 2002] on spread of epidemic disease on networks to the propagation of the financial malaise on the network of investment institutions. We also model and compute the financial malaise transmissibility as a function of the mean of the malaise spread time. The theoretical framework espoused in this paper explains the emergent behavior of the financial system of systems, namely, the financial malaise discussed in this paper. In particular, the theoretically obtained result (4) for the case of α = 2 and κ = 3 reasonably approximates the growing trend of the number of failed banks during the months between 2005 and the early part of 2009. Furthermore, the analytical approach can also allow systems engineers to extend their domain of expertise to systems of systems other than those they design or those that are in the engineering realm.

ACKNOWLEDGMENT Thanks to two reviewers, the connection of the approach employed in this paper to systems engineering has been explicitly made. References Abramowitz, M. and Stegun, C. A. (Eds.). 1964. Handbook of Mathematical Functions. Amaral, L. A. N., A. Scala, M. Barthelemy, and H. E. Stanley, "Classes of behavior of small-world networks," cond-mat/0001458. Atlas, J. and Dreier, P. 2007. The Conservative Origins of the Sub-Prime Mortgage Crisis/Everything you ever wanted to know about the mortgage meltdown but were afraid to ask. Web only, http://www.prospect.org/ Credit Crisis. 2008. Collateralized Debt Obligations (CDOs) and the Start of the Credit Crisis.

http://bonds.about.com/od/derivativesandexotics/a/CDO.htm FDIC. 2009. Failed Bank List. 2009. http://www.fdic.gov/bank/individual/failed/banklist.html Huynh, T.V. 2008. Orthogonal Array Experiment in Systems Engineering and Architecting ─ An Overview. Submitted to Systems Engineering and revised. Huynh, T.V. and J.S. Osmundson. 2008. Scrambling on Electrical Power Grids. Proceedings of the Third International Conference on System of Systems Engineering, Monterey, CA, 2-4 June, 2008. 978-1-4244-2173-2/08. Isidore, C. 2008. New recession worry: Bank failures, CNNMoney.com, March 3 2008. http://money.cnn.com/2008/03/03/news/economy/bank_failures/index. htm?postversion=200803031. Meyers, L.A., M.E.J. Newman, and B. Poubohloul. 2006. Predicting epidemics on direct contact networks. Journal of Theoretical biology 240(2006) 400-418. Meyers, L. A. 2007. Contact network epidemiology: bond percolation applied to infectious disease prediction and control. Bulletin (New Series) of the American Mathematical Society, Volume 44, Number 1, 63-86. Mysmp Admin. 2008. Collateralized Debt Obligation - CDO, http://bonds.about.com/od/derivativesandexotics/a/CDO.htm Newman, M. E. J., S. H. Strogatz, and D. J. Watts. 2001. Random graphs with arbitrary degree distributions and their applications. Phys. Rev. E 64, 026118. Newman, M. E. J. 2002. Spread of epidemic disease on networks. Physical Review E 66, no. 1, art. no. 016128. MR1919737 (2003e:60223). Newman, M.E.J. 2000. The structure of scientific collaboration networks. Santa Fe Institute working paper 00-07-037. Moore, C. and M. E. J. Newman. 2000. Exact solution of site and bond percolation on small-world networks. Phys. Rev. E 62, 7059-7064. Newman, M.E.J. 2001, Exact solutions of epidemic models on networks. (Dated: November 28, 2001). Osmundson, J.S., Langford, G.O, and Huynh, T.V. 2008. Emergent Behavior in an Unregulated Financial System of Systems: Economic Meltdown. Submitted to IS2009, Singapore Roubini, N. 2008. The Rising Risk of a Systemic Financial Meltdown: The Twelve Steps to Financial Disaster, RGE Monitor, February 5, 2008. http://www.rgemonitor.com/blog/roubini/242290 Sage, A. P. 1992. Systems Engineering. John Wiley & Sons, Hoboken, NJ, 1992. The Economist. 2008. When fortune frowned. A special report on the world economy. October 9, 2008. Wikipedia. 2008. http://en.wikipedia.org/wiki/Collateralized_debt_obligation#Conce pt

12

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 13 of 13

BIOGRAPHIES Thomas V. Huynh is an associate professor of systems engineering at the Naval Postgraduate School in Monterey, CA. His research interests include uncertainty management in systems engineering, complex systems and complexity theory, system scaling, simulation-based acquisition, and system-of-systems engineering methodology. Prior to joining the Naval Postgraduate School in 2003, he was a Fellow at the Lockheed Martin Advanced Technology Center in Palo Alto and Sunnyvale, CA, where he engaged in research in computer network performance, computer timing control, bandwidth allocation, heuristic algorithms, nonlinear estimation, perturbation theory, differential equations, and optimization. While he spent 23 years in the aerospace industry, he was also teaching part-time in the departments of Physics and Mathematics at San Jose State University. Dr. Huynh obtained simultaneously a B.S. in Chemical Engineering and a B.A. in Applied Mathematics from UC Berkeley and an M.S. and a Ph.D. in Physics from UCLA. Dr. Huynh is a member of INCOSE.

John S. Osmundson is an associate research professor with a joint appointment in the Systems Engineering and Information Sciences Departments at the Naval Postgraduate School in Monterey, CA. His research interest is applying systems engineering and computer modeling and simulation methodologies to the development of systems of systems architectures, performance models, and system trades of time-critical information systems. Prior to joining the Naval Postgraduate School in 1995, Dr. Osmundson worked for 23 years at Lockheed Missiles and Space Company (now Lockheed Martin Space Division) in Sunnyvale and Palo Alto, CA, as a systems engineer, systems engineering manager, and manager of advanced studies. Dr. Osmundson received a B.S. in physics from Stanford University and a Ph.D. in physics from the University of Maryland. Dr. Osmundson is a member of INCOSE.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 9

Advising Policy Makers – A Bayesian Approach Michael A. Duffy, Ph.D. National Renewable Energy Laboratory 1617 Cole Boulevard, MS-301 Golden, CO 80401-3393 303-275-3022 [email protected]

Copyright © 2009 by Alliance for Sustainable Energy, LLC. All Rights Reserved. This work is subject to Government rights. Used by APCOSE with permission for the 2009 APCOSE, International Council on Systems Engineering.

Abstract. Policy makers are often faced with making critical decisions under considerable uncertainty. Furthermore, the analyst conducting the analysis is generally not the same person as the policy maker. Thus, it is necessary to understand the competing paradigms between the scientific and policy making communities. The scientific community tends to avoid Type 1 errors (i.e., systems engineers don’t like to give bad advice, false positives), whereas policy makers tend to avoid Type 2 errors (i.e., preferring to hedge against a potentially catastrophic event, false negatives.)

Policy Decision Forecast Proves False Forecast Proves True

Accept Forecast: Type 1 error: Correct Decision Policy response follows Squandered resources

Reject Forecast Correct Decision Type 2 error: No policy response Unmitigated damages

In an effort to improve the reliability of intelligence reports and thus avoid both Type I and Type II errors, the recently passed Intelligence Reform and Terrorism Prevention Act (U.S. Government, 2004) calls for the use of analytic methodologies in support of future intelligence gathering activities. This paper proposes an analytic methodology based on Bayes Theorem which explicitly considers the level of Type I and II errors. The proposed methodology is retrospectively applied to the case of whether Iraq possessed weapons of mass destruction (WMD) prior to March 2003. Introduction In response to the September 11, 2001 terrorist attacks and the possibility that Iraq had weapons of mass destruction, the Intelligence Reform and Terrorism Prevention Act (U.S. Government, 2004) was passed by the United States Congress and signed by the President. Based on suggestions from the 9/11 Commission’s report (U.S. Government, 2004) and the Commission on the Intelligence Capabilities of the United States Regarding Weapons of Mass Destruction (U.S. Government, 2005), this new law specifically calls for the use of “analytic methodologies.” The question is how to make reasonable decisions in the face of uncertainty with respect to the true state of nature. Any analytic methodology that claims to improve the decision process should involve the

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 9

following steps (Morris, 1968): 1. Recognize and conceptualize the decision problem using both past experience and available data 2. Make and implement the decision 3. Learn form the results of the decision how it should be modified and add these results to the reservoir of experience on which future decisions may draw. Bayes theorem provides a tool for consistently combining new information with a decision maker’s prior experience and judgments. It helps ensure that lessons learned from one experience will lead to improved subsequent decisions. Bayesian Analysis Bayes' theorem relates the conditional probabilities of events A and B, where B has a non- vanishing probability, P(B│A) PR(A) PO(A│B) = PR(B) where: • PR(A) is the prior probability of A; i.e., it does not take into account any information about B. • PO(A|B) is the posterior probability of A, given B; i.e., it is conditional on the specified value of B. • P(B|A) is the conditional probability of B given A. • PR(B) is the prior probability of B. Bayes' theorem essentially describes the way in which one's beliefs about observing 'A' are updated by having observed 'B'. By doing so, it provides a consistent theory of learning. Hypothesis Testing Oftentimes, policy makers are asked to make decisions under considerable uncertainty. These decisions can often be framed in terms of two hypotheses: a "null hypothesis, H0 " which corresponds to a presumed default "state of nature" and an "alternative hypothesis, H1" which corresponds to the opposite situation. The null hypothesis, H0, represents a theory that has been put forward, either because it is believed to be true or because it is to be used as a basis for argument, but has not been proved. The decision maker must decide whether the null hypothesis should be discarded in favor of the alternative. Additional data may be gathered to support this decision. If the data does not correspond with the actual state of nature, then an error has occurred and the wrong decision may be made; but if the data corresponds with the actual state of nature, then the correct decision may be made. Since the null hypothesis relates to the statement being tested, we either "reject H0 in favor of H1" or "do not reject H0". If we conclude "do not reject H0 ", this does not necessarily mean that the null hypothesis is true, it only suggests that there is not sufficient evidence against H0 in favor of H1. Rejecting the null hypothesis then, suggests that the alternative hypothesis may be true.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 9 Type I and II Errors It is common in policy analysis to assign one of the following error types to wrong decisions: • Type I errors (the "false positive"): the error of rejecting the null hypothesis given that it is actually true; e.g., assuming Iraq does not possess WMD when in fact they do • Type II errors (the "false negative"): the error of failing to reject the null hypothesis given that the alternative hypothesis is actually true; e.g., assuming that Iraq possesses WMD when in fact they don’t.

• H0: Iraq possesses WMD

• H1: Iraq does not possess WMD

Table 1: WMD Decision Matrix Decision True State of Nature

H0 True H0 False Iraq possesses WMD Iraq does not possess WMD

Reject H0 Type I Error Correct Decision

Do not Reject H0 Correct Decision Type II Error

The scientific community tends to avoid Type 1 errors (i.e., systems engineers don’t like to give bad advice), whereas policy makers tend to avoid Type 2 errors (i.e., preferring to hedge against a potentially catastrophic event.) Weapons of Mass Destruction and the Iraq War After the 9/11 terrorist attacks, the U.S. Intelligence Community was charged with gathering intelligence about Iraq’s nuclear, biological, and chemical weapons programs. The Community’s best assessments were set out in an October 2002 National Intelligence Estimate, entitled, Iraq’s Continuing Programs for Weapons of Mass Destruction (see U.S. Government, 2005), specifically concluding that Iraq: • had reconstituted its nuclear weapons program and could assemble a device by the end of the decade • had biological weapons and mobile facilities for producing biological warfare agents • had both renewed production of chemical weapons, and probably had chemical weapons stockpiles of up to 500 metric tons • was developing unmanned aerial vehicles probably intended to deliver biological warfare agents.

These assessments were all wrong; in essence, they each represented a false positive, indicating the null hypothesis should not be rejected even though the alternative hypothesis turned out to be true. Table 2 contains definitions of the terminology and the assumptions that

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 9

will be used in a retrospective application of Bayesian analysis.

Table 2: Nomenclature Terminology H0 : Iraq possesses WMD H1 : Iraq does not possess WMD P: Intelligence evidence supports H0 (positive intelligence evidence) N: Intelligence evidence does not support H0 (negative intelligence evidence) Assumptions

1. The administration’s prior probability that H0 is true PR(H0)

2. The administration’s prior probability that H1 is true PR(H1) = 1 - PR(H0)

3. Probability of (Negative Evidence│WMD) P(N│H0) Type I Error

4. Probability of (Positive Evidence│WMD) P(P│H0) = 1-Prob (Type I Error)

5. Probability of (Positive Evidence│No WMD) P(P│H1) Type II Error

6. Probability of (Negative Evidence│No WMD) P(N│H1) = 1-Prob (Type II Error) Bayesian Revisions

7. The administration’s posterior probability that H0 is true, given positive evidence is received

PO(H0│P) = P(P│H0) PR(H0)

P(P│H0) PR(H0) + {P(P│H1)(1-PR(H0)}

8. The administration’s posterior probability that H0 is true, given negative evidence is received

PO(H0│N) = P(N│H0) PR(H0)

P(N│H0) PR(H0) + {P(N│H1)(1-PR(H0)}

Based on prior experience with Iraq, the administration would have formulated a judgment about whether Iraq possessed WMD, or not. Let us assume the following two cases: • Case 1: the administration is quite convinced that Iraq possessed WMD and thus assigned a 90% probability to the null hypothesis, PR(H0) = 0.90. • Case 2: the administration had no idea whether Iraq possessed WMD, or not, and thus assigned a 50% probability to the null hypothesis (representing the greatest possible uncertainty), PR(H0) = 0.50.

Furthermore, let us assume that the administration had a great deal of confidence in previous intelligence gathering efforts, and thus, assigned the following minimal errors to the intelligence reports: • Type I Error = 0.10, the probability that an intelligence report would result in a false positive • Type II Error = 0.20, the probability that an intelligence report would result in a false negative.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 9

Table 3 contains the results of revising the administration’s belief in the null hypothesis by applying Bayes theorem under these assumptions given four positive intelligence reports, each supporting the null hypothesis. If the administration had started with a 90% belief that Iraq possessed WMD before any additional information was provided, even just two positive reports would increase the administration’s belief to over 99%. Alternatively, if the administration initially had no idea of whether Iraq possessed WMD or not (i.e., Case 2), four positive intelligence reports would still increase the administration’s belief to over 99%.

Table 3: Bayesian Analysis: All Intelligence Reports Are Positive (Type I Error = 0.10; Type II Error = 0.20)

# of 1 2 PR(H0) PR(H1) P(P│H0) P(N│H0) P(P│H1) P(P) P(N) PO(H0│P) Positive Type I Type II Intelligence Error Error Reports

Administration’s Prior Probability PR(H0) = 0.90 1 0.90 0.10 0.90 0.10 0.20 0.83 0.17 0.9759 2 0.9759 0.0022 0.90 0.10 0.20 0.8831 0.1169 0.9945 3 0.9945 0.0005 0.90 0.10 0.20 0.8962 0.1038 0.9988 4 0.9988 0.0001 0.90 0.10 0.20 0.8991 0.1009 0.9997

Administration’s Prior Probability PR(H0) = 0.50 1 0.50 0.50 0.90 0.10 0.20 0.55 0.45 0.8182 2 0.8182 0.1818 0.90 0.10 0.20 0.7727 0.2273 0.9529 3 0.9529 0.0471 0.90 0.10 0.20 0.8671 0.1329 0.9891 4 0.9891 0.0109 0.90 0.10 0.20 0.8924 0.1076 0.9976 1 P(P)=P(P│H0) PR(H0) + P(P│H1)PR(H1) 2 P(N)=P(N│H0) PR(H0) + P(N│H1)PR(H1)

Figure 1 illustrates how quickly the administration’s belief would have increased with each subsequent positive intelligence report. Even if the administration had begun with the greatest possible uncertainty (i.e., PR(H0) = 0.50), the first two reports alone would have increased their belief in the null hypothesis to over 95%. Clearly, there is a need to increase the reliability of intelligence reports.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 9

Bayesian Update of Administration's Belief

1.00 0.80 0.60 0.40 0.20 Posterior Probability Posterior 0.00 01234

# of Positive Intelligence Reports

Figure 1. Bayesian Analysis Assuming All Positive Intelligence Reports (Type I Error = 0.10; Type II Error = 0.20) Had the intelligence reports correctly reported that there was no indication that Iraq possessed WMD, a Bayesian analysis would have yielded the revised beliefs in Table 4. If the administration had started with a 90% belief that Iraq possessed WMD before any additional information was provided, even just three negative reports would have decreased the administration’s belief to less than 2%. Alternatively, if the administration initially had no idea of whether Iraq possessed WMD or not (i.e., Case 2), the first two negative intelligence reports would have decreased the administration’s belief to less than 2%.

Table 4: Bayesian Analysis: All Intelligence Reports Are Negative (Type I Error = 0.10; Type II Error = 0.20)

# of 1 2 PR(H0) PR(H1) P(P│H0) P(N│H0) P(P│H1) P(P) P(N) PO(H0│P) Negative Type I Type II Intelligence Error Error Reports

Administration’s Prior Probability PR(H0) = 0.90 1 0.90 0.10 0.90 0.10 0.20 0.83 0.17 0.5294 2 0.5294 0.90 0.10 0.20 0.5706 0.4294 0.1233 3 0.1233 0.90 0.10 0.20 0.2863 0.7137 0.0173 4 0.0173 0.90 0.10 0.20 0.2121 0.7879 0.0022

Administration’s Prior Probability PR(H0) = 0.50 1 0.50 0.50 0.90 0.10 0.20 0.55 0.45 0.1111 2 0.1111 0.90 0.10 0.20 0.2778 0.7222 0.0154 3 0.0154 0.90 0.10 0.20 0.2108 0.7892 0.0019 4 0.0019 0.90 0.10 0.20 0.2014 0.7986 0.0002 1 P(P)=P(P│H0) PR(H0) + P(P│H1)PR(H1) 2 P(N)=P(N│H0) PR(H0) + P(N│H1)PR(H1)

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 9

Figure 2 illustrates how quickly the administration’s belief would have decreased had each intelligence report correctly not supported the null hypothesis. Even if the administration had begun with near certainty that Iraq did possess WMD (i.e., PR(H0) = 0.90), the first two reports alone would have decreased their belief in the null hypothesis to less than 2%.

Bayesian Update of Administration's Belief

1.00

0.80

0.60 0.40

0.20

Posterior Probability 0.00 01234 # of Negative Intelligence Reports

Figure 2: Bayesian Analysis Assuming All Negative Intelligence Reports (Type I Error = 0.10; Type II Error = 0.20)

It is interesting to note that if the administration’s prior probability [PR(H0)] had been either 1 or 0, then the posterior probability [PO(H0│evidence)] would be the same, no matter what evidence was provided to the administration (see items 7 and 8 in Table 2.) Consequently, if there is any possibility of changing one’s mind, prior probabilities of 1 or 0 should be avoided. Oftentimes, it is quite difficult to determine beforehand the values of Type I and Type II errors. In such cases, it may be easier to think in terms of a likelihood ratio (L) defined as the probability of a positive intelligence report, given that the null hypothesis is false, divided by the probability of a positive intelligence report, given that the null hypothesis is true:

P(P│H1) Type II Error L = = P(P│H0) 1 - Type I Error

Figures 3 and 4 illustrate the dependence of the posterior probability on the value of L as a function of the number of positive intelligence reports for PR(H0) = 0.90 and 0.50, respectively. The likelihood ratio is a good indicator of how decisive a particular intelligence report may be. Note that if it is equally likely to obtain a positive report whether the null hypothesis is true [P(P│H0)] or the alternative is true [P(P│H1)], then L=1 and the posterior probability equals the prior probability. Alternatively, in the unlikely case that L is greater than 1, a positive intelligence report would result in a posterior probability that would be less than the administration’s prior probability. The preferred situation, of course, is to minimize both Type I errors and Type II errors and, thereby, minimize the likelihood ratio.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 9

Bayesian Update as a Function of L

1.00

0.80 L=0.10 0.60 L= 0.50 0.40 L=1.0 L=2.0 0.20

Posterior Probability 0.00 01234 P(P│H1) # of Positive Intelligence Reports L = P(P│H0)

Figure 3. Bayesian Analysis as a Function of Likelihood Ratio (Assuming PR(H0) = 0.90)

Bayesian Update as a Function of L

1.00 0.80 L=0.10 0.60 L=0.50 0.40 L=1.0 L=2.0 0.20

Probability Posterior 0.00 01234 P(P│H1) # of Positive Intelligence Reports L = P(P│H0)

Figure 4. Bayesian Analysis as a Function of Likelihood Ratio (Assuming PR(H0) = 0.50)

Summary and Conclusions The U.S. Congress has recognized the need to implement more robust analytical methodologies into the intelligence gathering efforts of the various U.S. agencies, especially when faced with critical decisions under uncertainty. A retrospective application of Bayesian analysis to the determination of whether Iraq possessed WMD, or not, has shown how new information can be integrated with prior experience in a consistent manner. While the Bayesian revision of prior beliefs has been shown to help policy makers act in a manner consistent with the logic of probability theory, the importance of reducing Type I and Type II errors cannot be over emphasized. As conscientious systems engineers, it is our duty to strive for minimal Type I errors, whereas punctilious policy makers should strive to minimize Type II errors. Other important Bayesian applications that come to mind are the new administration’s potential response to Iran’s possible possession of a nuclear weapon and global climate change.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 9 References Blair, Bruce. 2004. The Logic of Intelligence Failure, Center for Defense Information. http://www.cdi.org/blair/logic.cfm. Garwin, Richard L. 2004. Science and National Intelligence. http://www.fas.org/rlg/040820- sani.pdf. Lee, Jennifer Lynn. 2007. Bayesian Reasoning Method For Intelligence Using Natural Frequencies, Masters Thesis, Mercyhurst College. Morris, William T. 1968. Management Science, A Bayesian Introduction, PRENTICE-HALL, Englewood Cliffs, NJ. United States Government. 2004. The 9/11 Commission Report. http://www.9- 11commission.gov/report/911Report.pdf. ———. 2004. Intelligence Reform and Terrorism Prevention Act of 2004, http://www.nctc.gov/docs/pl108_458.pdf. ———. 2005. Commission on the Intelligence Capabilities of the United States Regarding Weapons of mass Destruction, http://www.wmd.gov/wmd_report.pdf. Biography Michael Duffy, Ph.D. is the Chief Systems Engineer within the National Renewable Energy Laboratory’s Systems Engineering and Program Integration office. He has a Ph.D. from the Ohio State University in Systems Engineering, an M.S. from Northeastern University in Engineering Management, a second M.S. from the Massachusetts Institute of Technology in Mechanical Engineering, and a B.S. from Tufts University in Mechanical Engineering. His background includes over 35 years of systems engineering experience as a consultant and Chief Systems Engineer in energy, safeguards and security, nuclear waste management, national defense, transportation, and space programs. Dr. Duffy has been a member of the International Council on Systems Engineering since 1992.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 17

A Practical Application of MBSE - The Automated Parking System

Philip Simpkins, CSEP Adam Kleinholz Vitech Corporation Vitech Corporation 2826 Stokely Hill 605 Pawleys Arch San Antonio, TX 78258 Virginia Beach, VA 23462 210-267-1152 803.644.8178 [email protected] [email protected]

Joe Maley Vitech Corporation 2070 Chain Bridge Rd., Suite 100 Vienna, VA 22182 703.883.2270 x232 [email protected]

Copyright © 2009 by Philip Simpkins. Published and used by APCOSE with permission.

Abstract. A real-world application of Model Based Systems Engineering (MBSE) is a great way to display the insight that MBSE can provide. The June 2008 INCOSE conference held in Utrecht, The Netherlands, provided an opportunity to illustrate how MBSE contributes to gaining systems engineering insight and leads to innovative real-world design solutions. INCOSE’s tool vendor challenge provided the opportunity to demonstrate the benefits of MBSE through a simply defined problem statement. Several systems engineering tool vendors participated in this event. All vendors applied their systems engineering approach, derived a solution, and presented their results within the tool challenge’s time frame. This paper also illustrates, through the tool vendor challenge, how MBSE may be leveraged by other system engineering practitioners for the purpose of extending their knowledge and skills. This paper will explore the main activities and key concepts that lead to an integrated and convergent solution, which MBSE consistently provides. MBSE uses fewer resources to improve product timeliness, increases the team effectiveness, and provides defensible results. Introduction A tool vendor challenge was part of the recent INCOSE conference held in Utrecht, The Netherlands from 16-19 June 2008. The tool challenge‘s objective was for each tool vendor to prepare and to present a system solution for an easily understood hypothetical problem. The problem statement provided the problem’s loosely defined system context and a list of system engineering work products. The work products were the means by which the tool vendor challenge would measure the breadth and technical depth of each participant’s system engineering and tool capabilities. An Automated Parking System was the subject of the tool vendor challenge, and the respective design teams were required to provide or demonstrate the following:

Compose the system specification Demonstrate requirements handling Define the system in its environment Define logical sub-systems, flows from sub-systems to the environment and flows between one sub-system to another

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 17

Define sub-system activities Address the particularities of the human sub-systems Describe the functional modes of the system and transfers between them Draw a timeline for the development of the system Demonstrate how the tool supports the system engineering activities.

The other vendors participating in this challenge were:

3SL Artisan Cognition Corporation No Magic PKM Solutions Sodius Telelogic UGS/Siemens Visure

One of the things that made this project very exciting and challenging was that the coordinator only provided the teams six days to prepare their solution.

Upon completion of the model, we, at Vitech, thought it would be beneficial if others could review the model and understand the means of its construction with the hope that other systems engineering practitioners would leverage the MBSE concepts captured in this model. With this in mind, our purpose is to describe our design and relate the application of the MBSE methodology that led to this particular solution. Each subsequent paragraph explores a MBSE topic. To gain the most from this paper, it is recommended to acquire the APS problem statement (INCOSE 2008 Tool Vendor Challenge Statement) and model. Afterwards, load the model into the MBSE software tool; read each section, then navigate and explore the model. For a copy of the model please download it from our website at http://www.vitechcorp.com . Problem Statement: An Overview As the challenge authors stated, the NotilHotels Parking Lot challenge problem statement is easily understood by system engineering and non-system engineering practitioners. With this is in mind, there are a few technical terms that many people may not be aware of that are essential in the overall solution. When they arise, we will attempt to define them in sufficient detail so as to not inhibit the readers’ understanding of the MBSE approach. This paper highlights the salient points of the model and the MBSE approach.

Problem Overview The problem starts by defining the client company, NotilHotels, a world-wide hotel chain headquartered in Paris and specializing in low-cost, high valued business hotels located in downtown metropolitan cities. NotilHotel’s issue is that each hotel offers some space for parking, but the cars of people working in nearby offices often consume the available parking spaces. These congested parking lots cause arriving guests to be unable to park their cars within easy walking distance of the hotel. In order to secure the allotted parking spaces for the Hotel guests, NotilHotels desires to develop an automated parking system that regulates the use of the limited parking space.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 17

The following sections describe the technical approach using MBSE and modeling techniques, as expressed in the software tool, which best capture the system solution.

A Brief Overview of MBSE Model Based Systems Engineering is the proven method and technical approach used to developed the Automated Parking System (APS) solution. Figure 1 illustrates the time frame associated with the various engineering activities leading to a system specification.

0. Define Need & System Concept Activity bars represent movement of “center 1. Capture & Analyze Orig. Requirements of gravity” of system engineering team. 2. Define System Boundary Concurrent engineering is assumed. 3. Capture Originating Architecture Constraints 4. Derive System Threads 5. Derive Integrated System Behavior

6. Derive Component Hierarchy 7. Allocate Behavior to Components SCHEDULESCHEDULE 8. Define Internal Interfaces 9. Select Design

10. Perform Effectiveness & Feasibility Analyses

11. Define Resources, Error Detection, & Recovery Behavior

12. Develop Validation Requirements/Validation Plans

13. Generate Documentation and Specifications

Figure 1. Essential Systems Engineering Activities Timeline

Each of the horizontal bars, as shown in Figure 1, represents concurrent systems engineering activities in time (moving from left to right). It is also important to note that the concurrent activities support one another, i.e., they are interdependent. Each of Figure 1’s activities is discussed in the following sections and how these activities were applied to the APS problem.

Using MBSE to define and model your system by means of a well-structured systems modeling language and maintaining the model in a powerful, accessible repository yields the following significant benefits:

1. Provides total system definition, traceability, and consistency • Provides a consistent means of communication • Provides improved management of design complexity 2. The work products (specifications, reports, etc.) are by-products of the disciplined approach employed by the engineering team • Engineers focus on the engineering, not document or artifact production • Improves quality and productivity 3. Various system representations are automatically generated based on the definition in the repository • Improves system integrity throughout the design process 4. Advanced analyses and trades on the system definition occur with unparalleled insight and perspective

The training course and definition guides go into much more detail on how to apply MBSE with the software tool respective to the systems engineering activities. To help understand

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 17 these activities, this paper consolidates and discusses the following key MBSE concepts: 1. Define Need and System Concept 2. Capture originating requirements • Evaluate a source document • Extract requirements 3. Define the system boundary 4. Define the system behavior and physical architecture 5. Allocate the behavior to the physical components 6. Verification and Validation Define Need and System Concept This system’s concept of operations assumes a fully automated parking lot entrance and exit system. The entrance system offers priority to hotel guests and will verify if the user is a hotel guest and allow entrance based on the availability of spaces. Parking will still be available to non-hotel guests but on an as available basis. The user will be notified of space availability situation and is expected to act accordingly. If allowed entrance, the user will park their vehicle. When it is time to leave the parking lot or garage the exit system will distinguish whether the user as a guest, maintenance personal, emergency personnel, or other and charge accordingly. If a user is a guest, they will be allowed to pay at the exit or allowed to leave if they have paid at the hotel.

The System operates in four modes and threads have been developed supporting each mode. These operational modes are:

1. Normal hotel guest access 2. Emergency vehicle access 3. Maintenance vehicle access 4. Non-hotel guest access

The system interacts with the hotel and the users wanting to park. This is depicted in Figure 2 below:

Figure 2. APS Concept

Figure 2 is a conceptual model of the APS as we conceived it. The Automated Parking System must interface with the hotel to determine if the person approaching the entrance is a guest or

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 17 not. And of course, the APS must interface with the person wanting to enter the parking lot. Capture & Analyze Originating Requirements The purpose of this activity is to capture and understand the customer’s requirements. The process which we follow is to capture the customer’s written text without alteration of the original text. If a block of text contains multiple requirement statements, we extract the individual requirement statements and create a requirement for each statement linked to the original composite source statement. This approach provides traceability between the original text and the decomposed expression of the originating text. This process is not a purely mechanical process because the engineer should also be reading the requirements looking for ambiguities, conflicts and gaps in the requirements. In doing so, the engineer then documents and identifies project issues and risks. Figure 3 shows how this activity, using MBSE software tool, Element Extractor, captures, structures the requirements and relates them to any uncovered issues, decisions and risks.

Figure 3 Capture & Analyze Process Flow

In this particular problem, we extracted 11 originating requirements, as shown in figure 3, and 9 issues. After the issues were documented, we did not stop there. For selected issues, we postulated alternative solutions for each alternative and made certain assumptions and decisions. Some of these documented decisions became the basis for derived requirements.

Figure 4 is a table view of the requirements extracted from the INCOSE problem statement and the derived requirements stemming from our requirements analysis and assumptions. Each requirement has an assigned: 1. Number, 2. Document Project Unique Identifier (PUID) and number/acronym which is automatically generated from a provided utility script,

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 17

3. Requirement description, 4. Requirement type, which delineates the requirement type such as functional, constraint, performance, verification or composite. 5. Origin attribute which delineates where the requirement‘s source.

Figure 4 APS Requirements

These are not all the attributes or relationships that are provided for requirements definition and management in the MBSE software tool. The problem statement asked to demonstrate requirements handling. The activity which was just described largely fulfills this objective. The other aspect of requirements handling that was not discussed was the ability to perform change impact assessment. Impact assessment is accomplished through the custom hierarchy feature. This feature allows you to select a set of interdependent relationships which when queried through the software tool results in displaying the relevant aspects of the model. Use of this custom hierarchy feature allows the engineer to quickly assess how proposed changes impact the model and the system design. Define the System Behavior and Physical Architecture

Define System Boundary Concurrent to capturing and analyzing the requirements, the engineer will discover external systems and interfaces. As system understanding evolves, the engineer identifies the system

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 17

boundary, which establishes the external interfaces for the APS. An example of this partitioning is deciding whether to include the user as part of the system or treat the user as external to the system. This is an engineering decision that the engineer must make and is not always cut and dry. In this particular model, we decided that the user is an external system because the user is utilizing the system and is not an integral part of its operations. Figure 5 is a physical block diagram of the APS Context; this diagram evolves as the requirements are understood and the architectural constraints are uncovered.

C.2 Hotel_User Registration Information

Users

External System

C.1

Hotel

Facility Fee Information Link APS_User Information Portal Operation_Enter_Link Portal User Notification Link Portal Control Link Exit Message Link Message Exit Portal Operation_Exit_Link Portal Entrance Portal User Information User Portal Entrance

SYS.1 Hotel_APS Information Reservation Data Link Automated Parking System Space Management Information System

Date: Author: 6/16/2008 Administrator Number: Name: C NotilHotel Parking Context

Figure 5 APS Context Diagram

Derive System Threads System threads form the essential building blocks that comprise a functional architecture. The essence of a thread is to understand the system’s black-box behavior through stimulus-response analysis. An APS example of such a thread is the mode of operation thread. As stated in the APS problem statement, we are required to demonstrate how the system is expected to operate in each mode, such as when a hotel guest enters the parking lot. If we describe the problem using text only, we would describe the thread as follows:

• Initial Conditions o Automated Parking System is operating correctly o Hotel computer is functioning correctly o User is registered at the hotel

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 17

• Flow of events o The user approaches the parking lot o The APS portal system detects a user wanting to enter the parking lot o The APS system requests the user’s identification o The user presents information that they are a registered hotel guest o The APS verifies the registration of the user o The APS allows the user to enter o The APS manages the space allotment o The user parks their vehicle • Final Conditions o The guest is allowed to enter o The entrance portal is ready to accept the next user

Using the software tool, we graphically model each thread using Enhanced Functional Flow Block Diagram (EFFBD) notation. The EFFBD is a robust executable notation describing system behavior. The graphical model provides the functional representations (white rectangular boxes); inputs/triggers, output representations (green rounded rectangles); and their logical placement on the graph using logical notations.1 Each EFFBD thread model is validated through simulation, functional timeline analysis, documentation and online reviews through group collaboration to achieve static and dynamic consistency.

t.1.4

t.1 Update Space Availability

t.1.1 t.1.2 t.1.3 APS AND AND t.1 Detect Guest t.1 Request ID t.1 Identify Guest

t.1.5 t.1 Allow Entrance to Parking t.1 t.1 ID Proximity ... Request t.1 ID t.1 Open Portal AND AND

t.1.6 t.1.7 t.1.8 User t.1 Approaches t.1 Provide t.1 Park Vehicle Parking Portal Identification

Date: Author: 6/13/2008 Administrator Number: Name: t.1 t.1 Guest Enters Parking Figure 6 User Enters the Parking Lot Thread

For a complete system functional definition, there should be a unique thread for each external stimulus into the system. For this particular system, we developed eight essential threads that were used to form the basis of the integrated functional architecture.

1 For a deeper understanding of the EFFBD notation and its ability to describe system behavior, please refer to the white paper by Jim Long (Long, 2002), which describes the attributes of EFFBD and its comparison to other common system notations.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 17

After modeling the eight threads, we conceived and developed the integrated functional architecture. Figures 7, 8 and 9 are snapshots of three of the eight behavioral threads that were used to develop the integrated behavior. These three models and the model above provide a general picture of the expected operation of the APS. The models are located in the model under the function class and within the subfolder called System Threads.

Figure 7 Paying Non Guest Enters

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 17

Figure 8 Guest Exits

Figure 9 Non-Guest Exits

Derive Integrated System Behavior Deriving the integrated system behavior commences after the essential system threads are defined. One might ask, “What are the essential threads?” Unfortunately, there is no recipe for determining the entirety of the essential threads. Our general approach is to understand the system’s goals and identify the one or more key threads that support achieving these goals.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 17

These essential system threads then form the basis of the integrated functional architecture. Another important observation about behavior modeling is to be aware of your modeling perspective. A model perspective is where one or more system observers interact with the system, such as, user, supplier, stakeholder, customer, or operator.

Integrated System Model Obtaining the integrated system model is the objective of the functional analysis effort. The integrated system model will become the basis for the expression of the functional requirements and will be presented in the APS Specification. We start with defining the functional context. This functional context captures the system, its externals and their respective inputs, outputs and triggers. The context model for the APS is shown in Figure 10 – Integrated System Model.

Figure 10 Integrated APS Model

As previously stated, the system threads serve as the raw materials for developing the integrated system behavior model. A complex system may have a very large number of unique threads and to develop all possible conditions and paths in the model is not practical. The practical approach is to identify the essential threads by critical function and through these critical functions identify key behavioral patterns.2 When these key behavioral patterns are interpreted and developed within the system’s operational context, behavioral modeling of complex systems is manageable. This concept leads to a general heuristic approach for developing an integrated system model. Once the general functional architecture is in place,

2 After each new thread is integrated, it is recommended to simulate the thread to ensure that the model remains executable.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 12 of 17 further model refinement is accomplished through decomposing the functional behavior in layers to gain system insight and understanding as well as knowing when to stop. Through the development process, the system engineer gains valuable insight about the requirements and the overall functional architecture. It is very common for the engineer using this approach to discover missing functions and requirements, or identify better functional allocations leading to minimizing interface complexity and maximizing shared information. This APS model provides an excellent learning opportunity for the diligent student of MBSE. The model provided is just one possible solution. As the model evolves, it is important to link the originating functional requirements with the functions in the model.

Derive Component Hierarchy The component hierarchy was derived largely from the functionality of the system. The decision to have the user operate as an external system drove the external system decisions and helped defined the system’s boundaries. The needed functionality of an entrance and exit portal, along with the need to track parking spaces, drove the sub-system decisions in this design. Figure 11 shows the component hierarchy of the system.

Figure 11 APS Component Hierarchy Allocate Behavior to Components

The allocation of behavior to components is the systems engineering activity that maps the integrated behavior to the components performing those functions as shown in Figure 12.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 13 of 17

Figure 12 APS Function Allocation Hierarchy

This allocation decision defines the requirements for the respective components. The allocation decision can be a complex undertaking and does depend on many factors. As a note, several of the functions were allocated to more than one component. This was due to the fact that several of the components shared common behavior, such as space availability. This occurrence happens quite often in system development and is a favorable outcome.

Define Internal Interfaces Up to this point, the discussion focused on the functional architecture and the necessary functions and items that were used to describe the system behavior. Once functions are allocated to components, the corresponding input and output relationships that now cross component boundaries identify our logical interfaces. The Physical Block Diagram as shown in Figure 13 captures the manner in which each system component is related to other system components (interfaces as expressed through links).

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 14 of 17

Figure 13 APS Physical Block Diagram

The software tool has two classes that support interface definition. These are the Interface and Link class. The Links and associated components are what are shown in the physical block diagram. As part of the model development, the system engineer must associate an Item that has an input/output relationship with corresponding functions allocated to two components with a Link that connects these same components. These relationships serve to bridge the logical behavioral domain with the physical architectural domain. Links are also defined for non-functional interfaces to express a physical interface between two components. Links not only address the relationship between the components and items, but they also model the transport time for moving an item across a link (time delays). The Link class allows us to incorporate delays to further refine our understanding of the system behavior.

Select Design Design choice depends upon available technology; hence in this case, implementation options are few. However in other cases, there are technology options that result in different solutions. These alternate paths may be managed in parallel projects. Design selection may be performance driven, if performance is not a deciding factor, then other factors, such as size, cost, and manufacturability, enter into the decision process. Verification & Validation

Perform Effectiveness & Feasibility Analyses The effectiveness of a logical design can be assessed by simulation to analyze such aspects as

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 15 of 17

throughput and timing. Feasibility analysis assures that the logical and physical design can actually be implemented using available technologies.

Define Resources, Error Detection & Recovery Behavior Most systems consume and produce resources, such as power, that impact their performance. Resource utilization is easily modelled in the MBSE software tool and used to determine the needed levels and availability of a resource.

When developing an integrated model, the “success” paths are addressed. The integrated model is not complete until the failure modes are identified and recovery logic (which can be a significant portion of the integrated logic) is incorporated. In our APS solution, we address the possibility of an emergency or maintenance vehicle entering the system.

Develop Verification Requirements/Verification Plans Verification requirements establish the criteria by which it will be confirmed that the system meets its requirements and the methods by which this will be done. The MBSE software tool supports this activity and the complete identification and documentation of the verification events, procedures, and test configurations. Generate Documentation and Specifications The APS package includes the System Description Document (a systems engineering oriented presentation) and a System/Segment Specification for the APS. These are generated directly from the tools repository. In addition to these documents, the MBSE software tool offers extensive reporting options covering engineering information, management information, specifications, test plans, tool exchange, and utility scripts. Conclusion By applying these Key MBSE concepts with the software tool, design and systemic value, development time and communication superiority is achieved as shown in Table 1.

Table 1: MBSE Improvements

Area Improvement by applying MBSE and the software tool

Force • More can be done with fewer resources via Multiplier – Automated documentation

– Automated input parsing

– Documentation templates

– Effort to produce custom reports is reduced

– Structure and methodology reduces dependency on “experts”

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 16 of 17

Timeliness • Faster response to stakeholder inquiries – Responds to questions quickly and correctly

Quality • Information value is enhanced through – Automated consistency checking

– Change control

Effectiveness • Engineers gain better insight through – Models and simulation

– Information relationships

Defendable • Answers are objective not subjective Results – Answers are supported through rigorous traceability

– Answers are accountable – Who ,What, When

References

Long, Jim. 2002. Relationships between Common Graphical Representations in Systems Engineering, White paper, Vitech Corporation. http://www.vitechcorp.com Wieringa, R.J. 2003. Design Methods for Reactive Systems, Morgan Kaufmann Publishers, ISBN: 1-55860-755-2 INCOSE 2008 Tool Vendor Challenge Problem Statement, “Disclosure of the Challenge”, 11 June 2008, NotilHotels Automatic Parking Lot. http://www.incose.org/symp2008/index.php?option=com_content&ta sk=blogcategory&id=68&Itemid=134

BIOGRAPHY Phil Simpkins is a Senior Systems Engineer with Vitech Corporation, the developers of the CORE® system engineering tool. With more than 18 years of collective experience in nuclear and chemical process operations and systems engineering and nuclear safety analysis, Phil Simpkins worked for the Department of Energy where he managed projects and supported mission-essential designs prior to joining Vitech in 2005. With extensive knowledge in systems engineering, functions and requirements analysis, risk management, interface management and analytical software, Simpkins has delivered keynote presentations to various professional societies, and Vitech’s CORE user group events. He remains active in the International Council on Systems Engineering (INCOSE), serving as 2008 Region V Member Board Representative, 2008 president of the Alamo Chapter in Texas, 2005 president of the Central Savannah River Chapter in South Carolina, as well as participating on several

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 17 of 17

international committees. A Certified System Engineering Professional (CSEP), he holds a bachelor’s of science degree in Electrical Engineering and a master’s degree in Engineering Management from the University of Missouri-Rolla.

Adam Kleinholz brings more than 30 years of experience to Vitech as a Principal Systems Engineer, responsible for assisting clients such as the U. S. Departments of Energy and Defense with CORE implementation, project management, requirements analysis and process improvements. Prior to joining Vitech, Kleinholz applied his systems engineering and project management expertise developed during his Naval career at the Savannah River Site, a major Department of Energy facility in the southeast. Kleinholz holds a bachelor’s degree in Nuclear Engineering from the University of Oklahoma and a master’s degree in Mechanical Engineering from the Naval Postgraduate School in Monterey, CA. Among his many accolades, he received the George Westinghouse Award for implementation of the Reactor Safety Improvement Program at the Savannah River Site and academic achievement awards from General Dynamics, the U.S. Naval Institute, as well as the Vice Admiral Bryan, Naval Sea Systems Command Award for his post-graduate academic work.

Joe Maley joined Vitech in 1996 and brings nearly 40 years of systems security engineering experience to the team as a Principal Systems Engineer. He helps clients such as the U.S. Department of Defense create secure systems and efficient and effective business processes. Areas of work include organizational redesign, secure networks, system security profiling and electronic key management where tools such as CORE have been applied to design and model complex systems. Throughout his career, he has worked with Booz, Allen, and Hamilton, UNISYS Corporation and others where he has become an expert analyst of Communications Security (COMSEC), Information Security (INFOSEC) systems and the application of information assurance principles. Maley has authored numerous publications, delivered keynote presentations and is a member of the International Council on Systems Engineering (INCOSE).

Multi-Culture Structure of Opera, “Back to the Origin” based on SE Process Sung Ki Min, CEO Chan Hae Lee, Professor Institute of systems engineering, Inc. College of Music, Yonsei Univ. Seoul, Korea Seoul, Korea +1-82-2-733-6447 +1-82-2-2123-3080 [email protected] [email protected]

Copyright@2009 by Sung Ki Min & Chan Hae Lee Published and used by APCOSE with permission.

This opera is not bound to a particular time and space, attempting to combine various cultures, which leads to refer to this opera as the multi- culture-structure. In this respect, it is a revolutionary work that shows a new way for the modern opera. To do that, Systems Engineering process is good to apply for the scope of work, high level architecture of opera and detail architecture of theme, etc. Practically it is applied to “Back to the Origin.”

Scope of work The drama of the opera dates back to the time when the tower of Babel was built. Because of the tower, according to Genesis, God confuses languages, resulting in misunderstanding and miscommunication among various nations. In the middle of this state of disorder, two groups of characters are placed. On the one hand, the first group, referred as the good and the spiritual, consists of Anna, Asa, and Sihon, who pursue the “truth” and “love”. On the other hand, the second group, including Alex, Ohola and Mirmah, who are struggling to gain “wealth” and “power”. This group is dubbed in this opera as “the wicked” and “the worldly”. Having setting up these two different groups of characters, the inherent conflict of these characters is drawn. For example, the characters, such as Alex and Asa, attempt to unify the languages that had been in the state of disorder since the fall of the Babel. However, the way each character tries to achieve this goal is different, from which the conflict of the opera begins to grow; whereas Alex tries to unify the confused languages by using his military power, Asa, who invented a machine to unify the languages, attempted to do so by relying on his own knowledge. To begin with, let us first consider the title of this opera. The concept of the term, the Origin, plays an integral role in unfolding the drama and music of the opera. In this work, by defining it as α, it is important to create an inclusive and fundamental concept that contains the meaning of source, foundation, and the very beginning. To these, the message of “life,” “truth,” and “love” shall be considered for a definition of “the Origin,” that goes beyond the boundary of Christianity. Subsequently, these concepts will occupy the most important dramatic topoi of this opera, constantly occurring throughout the entire work. Despite the fact that this libretto is partly based on the Old Testament, the libretto is also appreciated by the modern audience because of the message of pursuing truth and love. These messages, in this regard, play roles that connects the seemingly remote message of the tower of Babel with the audience who lives in the 21st century. However, the question of how to musically as well as dramatically connect the story of Genesis with the modern audience remains as a compositional challenge that the composer is willing to cope with. To deal with the challenge, this work employs the role of a baby who suffers from illness and a dancer who is silent throughout the entire drama, both representing the fundamental and universal messages of truth and love. The baby, while also representing “life,” needs care and love while the dancer always leads us to draw our attention to the baby through his/her dance gesture. In order words, the silent dancer realizes what needs to be done in order to help the baby, but is not able to speak. Instead, he/she tries to tell the significance of sympathy and love in the form of his/her silent dance. Furthermore, the fact that both characters are silent implicitly underscores what the construction of Babel causes to the humanity, namely, the confusion (or the state of disorder) of languages, an important topoi that is dealt with in the opera. Scenario of truth, First, the truth, the scientist Asa who seeks for God’s guidance; the nationalist, Sihon who seeks the truth with the way that is contrast to Asa’; Anna who is passive and accepts her destiny Second, Alex struggles to gain power and Mirmah and Oholah who seek for the worldy. By illustrating the contrasting two dimensions of the libretto, the opera further underlines the significance of truth.

2 HHHighHigh LLLevelLevel AAArchitectureArchitecturerchitecturerchitecture of the OOOperaOperaperapera Instead of planning the musical structure of the opera in a linear, chronological way, it was started to work with the most important and fundamental parts of the opera, namely, Scene III and XI, from which the use of multimedia is found. Scene III appears before the complex drama of the opera begins to unfold, while Scene XIII marks the musical and the dramatic climax of the opera. Keeping their significance in mind, it divides the opera into 4 large sections. These are: - Section A: Exposition (music)/ drama (prologue) - Section B: Development (music and drama) - Section C: Recapitulation and elimination (music and drama) - Section D: Epilogue (music and drama) A comment is required in respect to Section A. As the above division indicates, unlike the remaining sections, the formal function of the music does not coincide with that of the drama, creating a musical exposition coexisting with a dramatic prologue. In other words, the music of Section A already encapsulates the musical themes that we will encounter in this opera. Meanwhile, the entire opera also can be divided into 16 musical units, which is marked from Part I to Part XVI, with each part sharing distinctive musical styles (and themes). In Section B and C, however, a number of parts groups together to form a larger musical unit. For example, Part III, IV, and V of Section B need to be recognized as one unit.

- Section A Part I & Part II: Part I and II are overlapped - Section B Part III + IV + V & Part VI + VII Part VIII + IX + X + XI + XII & Part XIII - Section C Part XIV + XV - Section D Part XVI Of the several parts that I showed above, Part XIII (Scene XI, On the Street) plays the most important role in planning the entire structure of the opera. For example, gradually and increasingly the musical and dramatic tension buil up toward Scene XI, only releasing it after the presentation of the scene.

3 AAArchitectureArchitecture of TThemehemehemeheme The theme of this opera is divided into two parts: dramatic theme and musical theme. Also, this opera can be divided into 12 scenes with 40 rehearsal numbers. As shown in Table 1 below, the theme would like to divide into 4 parts, namely, Exposition, Development, Recapitulation (Elimination), and Epilogue. It is further divided into 16 parts. In addition, the dramatic theme of the opera can be divided into two parts while the musical theme can be seen to have 10 sub-themes. As the below table shows, the opera grows increasingly in its musical structure and reaches its climax in the “On the street” scene in which it delivers explosive dramatic power in order to effectively express the main message of the opera. Table1. The architecture of theme

Dramatic Theme Dramatically, the most important scene is Scene XI, on the street. It takes place after Alex’s attempt to unify already confused language by activating a machine. He failed to do so, only creating further confusion. To express the state of confusion, instead of destroying the syntax of the given text, it

4 employs various and different languages piled up together, thereby expressing misunderstanding and miscommunication. In this chaotic state, Ana is searching for help, which, because of the miscommunication among people, turns out to be unsuccessful. The scene, reinforced by Ana’s crying, leads us to the state of despair that is resulted from the disconnection and not a physical, but psychological distance between Ana and the other characters. Musically, Scene XI, on the Street, provides us with various musical features. For instance, it includes Sprachstimme, the origin theme, and multimedia, interwoven together to illustrate the disorder caused by the confusion of languages. Likewise, these aspects, when piled up together, highlights the message of the opera, love and trust, which is needed to recover the state of disorder. “On the Street,” in this regard, occupies the center of the opera in which the dramatic and musical climax coincides.

1) Origin theme

In this part, what the composer have achieved in order to create a work that is universal in its drama and music. In order to cope with the conflict, it employs a dancer who is silent throughout the entire work. The gesture created by the dancer, then, was derived from that of Korean traditional dance. As a result, his/her gesture during the opera was linear, static, and understated, further emphasizing the significance of the role of the dancer. Along with the dancer, it also employs a Korean transverse bamboo flute (Daegeum) and a soprano solo (Chang), both playing tunes, which are derived from the Korean traditional melodies. In this work, however, the composer leaves the choice of the instrument and the tune to a performer who, then, employs a traditional instrument and melody from his/her own native culture. In turn, depending on where the opera is presented, the instrument can be varied and the tune can be improvised as long as they adhere to the traditional sources. To the dancer and the flute, it also adds a soprano solo to the opera whose melody also comes from Korean traditional tune. Similarly, this tune also can be improvised depends on where the opera is performed. Meanwhile, it incorporates a multimedia into the opera, presenting the visual images accompanied by the electronic music. By considering these two as a unit, it identifies them as the “visual and aural theme,” of the opera. Similar to the origin theme that includes three elements, the visual and aural theme only takes place both at the same time. More importantly, it asserts that

5 the use of the multimedia is one of the most effective tools to be appreciated by the modern audience, refusing to use it as the peripheral aspects of the opera only supporting the unfolding of the work.

2) Visual & aural theme

The visual and aural theme is overlapped with the presence of the origin theme. As a result, it creates a multi-layered work that combines the traditional with the modern aspects in this work as a means of achieving a sophisticated musical as well as dramatic structure that freely moves between time and space. The music of the opera is put together to divide a large musical structure, it will address two, the most innovative scenes of the opera, that is, Scene III and XI, in which the visual and the aural theme is present. These two scenes, more importantly, occupy the most important musical as well as dramatic climax of the opera. The first scene, Scene III, is directly related to Genesis, Chapter 11, verse 1 to 10, where God confuses people’s language. The second scene, Scene XIII, takes place after Alex’s attempt to unify the confused language using the machine invented by Asa. In Scene III confuses God peoples’ language. The scene deals with the consequence of building the Babel. In this scene, God’s wrath is presented in the form of the confusion of people’s language, resulting in no communication and understanding among people. Today’s communication network can be understood as an outcome of Babylonian times, which represents the hidden symbol of current speech. Therefore, Babylonia may be the original cause for the development of fear. The complex use of symbols and the longing to return to the basic origin leaves one in a state of hollowness, separating our grounds and leaving no place to stand. To depict the confused languages, along with the visual image, it created a libretto that illustrates the state of disorder and confusion by intentionally destroying the syntax of the given text. The music for the moment, then, was presented in the form of Sprachstimme.

3) The relationship between the characters and structure

a) Anna, Asa & Sihon

This work presents a structure that is not linear, but multi-layered, which is manifested both in its characters and drama. Let us conisder Anna, Asa, Sihon,

6 for example. In terms of pursuing truth, they have the same goal. However, each has his/her own character, thereby having a different way of achieving the goal. Asa seeks for truth, refusing to go against God. In contrast to Asa, Sihon is a nationalist who explores variosu ways of achieving his goal. Anna, on the other hand, also seeks for truth, but is fragile and passive. These three characters seek for and accept truth in different ways. Asa Anna Sihon

b) Alex, Ohola & Mirmah

Alex, Ohola, Mirmah share the same goal and the same way of achieving it in pursuing power and wealth. They value the wordly and do anything to acheive their goals. At the same time, they also believe that Alex who represents the institution can unite the world. Ohala Alex Mirmah

c) Vertical structure

In the scene of conflict in conversation, it places two different sets of conversation taking place both at the same time; first the conversation between Alex and Mirmah and second between Anna/Asa. In other words, the two different groups of characters are presented simultaneously on the stage, further underscoring the fact that each group pursues the different goal. Moreover, the two sets of conversation move to the three sets, including the conversation among Alex/Mirmah, Ohola/Sihon, and Asa/Anna occupying the three different places of the stage to represent the hierarchical structure of the characters. d) Horizontal structure

In Alex’ office, the composer highlights Alex. The red light, Light One, alternates with the neutral light, Light Two, highlighting Ohola and Alex and Alex in order to show two different dimensions of Alex’s character. Ohola Alex Alex Mirmah Moreover, the conversation between the two groups is further emphasized by the effect of the lights, further adding confusion to the opera.

7 e) Cross structure

The silent baby and the woman, in fact, are the protagonist of the opera. They appear in the begining, middle, and the end of opera. The fact shows great contrast to the conflict of the drama, which is caused by the confusion of languages. The six characters to solve the conflict by unifying language completes the opera by baby’s dying. The silent woman, who witness the entire drama, expresses her sympathy, leads the audience to find truth. The Silent Woman Asa Anna Sihon The baby The baby symbolizes the origin of life, and the silent woman represnets the redemmer. In this respect, they present a vertical relationship, which is connected by Anna. Musical Theme

1) Part I-II: primitive theme, noise theme and tower theme

a) Primitive Theme: If the two themes above are seen in a dramatic sense, the 10 themes below need to be seen in a musical sense. The opera begins with two percussion instruments whose motive resembles a stong heart beat. The themes refer to the Primitive Theme. The synthesizer, timpani, and side drum add more energy to this theme. The theme is presented at the beginning of the opera, then reappearing “On the Street” scene. Here, it varies the theme, but its presence is underscored by the powerful rhythm it plays, which, then, leaves a strong impression to the audience. This theme can be referred as the main theme of the opera becuase it signifies the Origin.

Table2. Part I-II

It presents the variation of a Korean traditional rhythm while in 2 the noise theme depicts the beginning of the world. The visual and aural images along

8 with the origin theme is presneted in 3 and 4. Here, three aspects of a long transverse bamboo flute, female solo, a dancer are also presented. In 6, we witness the appearance of a female ensemble (representing the worldly/wicked who presents the ascending chromatic lines (resembling the act of building the tower). In 7, the opera presents four anonymous characters in purple, gold, blue and white, representing power, wealth, wisdom, and truth respectively. 7 is followed by the female ensemble, presenting the tower theme. A choir takes on the tower theme, gradually building musico-dramatic tension in 7 + 8. Here, 7 and 8 form one musical unit. The Piu mosso functions as a bridge to Scene Ⅲ, creating a very abrupt and unexpected shift. It also Includes a part of the confuse theme that recurs as the worldly/wicked characters appear. In 9, Scene Ⅲ, God confuses people’s language. This scene represents the state of disorder, disturbance and confusion expressed by a destroyed syntax, while building up musico-dramatic tension by gradually adding voices. A pause with the lowest voice presents an understated yet strong statement, while completing the scene with spot (ha ha ha) motive, which is derived from the confuse theme. A part of the confuse theme leads to and completes Scene Ⅲ, thereby highlighting the significance of the scene. The composer creates the music completely corresponding to the drama.

b) Noise Theme (Chaos theme): The instruments employed in this section play the high register with the busy passages that represents disorder. The drama at this point describes how the earth was formed. It tells us the chaotic state of the earth that precedes God’s creation. I refer to this theme as the Chaos or Noise theme. This theme is presneted in the sixteenth note triplet figurations. It stops in the millde, then, transforms into the visual theme. The breath that is trying to announce an important message is accompanyed by the origin theme. After presenting the origin theme, the noise theme reappears, featuring faster, busier rhythm. It is again overlapped with the tower theme that represents power and challenge. The chaos theme, which was originally conceived as a representation of chaotic stage of the earth, is reused along with the presence of Alex. This, in turn, foretells the complete chaos that is taking place on the ”On the street” scene.

9 c) Tower Theme: The theme, sung by female ensemble which symbolizes three evils has a rising scale. Simultaneously it also shows us visual tone painting using the formation of laying bricks in the harmonic aspect. This theme suggests power, wealth, and desire. The theme usually appears in the scene that shows arrogance of human. Especially all the characters on the stage sing the theme and it gets connected to the verbal chaos. Tower theme appears in between the appearance of Alex as a symbol of power and functions as a leitmotiv.

2) Part III-V: each character displays his/her own motive. For example, Anna, representing a figure that stands at the closest to the message of this opera, is always portrayed with a consonant, smooth, and conjunct melody and harmony. On the other hand, Alex’s music is full of angular, harsh, and disjunctive intervals, separating him from the other characters. Likewise, the harmony that accompanies Alex’ music is always highly dissonant. She also assigned relatively high register for Sihon, the active nationalist, whose music is often accompanied by a tone-cluster. By doing this, she create a musical person who is struggling between his ideal and real world.

Table3. Part III-V

It reveals the systematic treatment of them in relation to each character which is not only the melodic configurations and harmonies, but also the meters that it employed for each character. In other words, it connected a character with a particular (set of) meter, constantly referring it as the character appears in the drama.

10

Alex’s Leitmotiv is vertically and horizontally simple, but presented along with strong accent. This, in turn, musically tells us that his character is straightforward but strong. Ohola’s Leitmotive includes a fast-moving triplet, It accented by the timpani who plays the diminished 5th. Thsi motive also describes her character as cunny and clever who has a stoing wordly desire. On the other hand, Mirmah’s Leitmotiv appears to show both musical characteristics that I pointed out above; it is simple, but is accented with fast- moving rhythm. This relates to the dramatic character which moves between Alex and Ohola. In particular, Scene.7 (Alex Castl) includes a conversation scene between Alex and Mirma that delivers who they are and what they represent through the instrumental accompaniment.

Section 16 presents the heart motive ( ) of pounding rhythmic figures, relating to the appearance of Sihon, Asa’s expression of anger. This motive best represetns Sihon’s temper.

11 3) Part VI-VII: From Part VI to Part VII, the drama of the opera begins to develop. Here, the wicked characters are all connected by the confuse theme (highly dissonant, angular, rapidly moving repetitive musical figures. And discord (conflict) among these characters is also present. However, these parts do relate the tower theme to the music of Anna and Asa. From 19 to 20, we observe the expansion of the musical structure. Here, the female ensemble is added, eventually leading to the chorus. Table4. Part VI-VII

a) Confuse- & unconfused theme: The confused- and unconfused theme which is shown above, portray the extreme conflict. These themes show the conflict between the characters who do not seem to have a common goal. Even in the scene where no conflict exists between the characters, the composer musically adds the inherent conflict. By using the confused theme, the composer hopes to communicate with her audience.

4) Part VIII-XII: In section 21, the opera features Mirmah’s music, which combines the Alex’s motive with the Ohola’s (Alex’s motive: short, dissonant, and agitated chord marked as opposed to Ohola’s continuous triplet figures ).

12 The confuse theme occurs when Mirmah reveals his plan in section 22. In section 23, Sihon’s less agitated chord takes on Alex’s. Table5. Part VIII-XII

5) Part XIII: As pointed out above, this scene reaches the musical climax of the opera. Part XIII brings in the entire themes and aspects presented earlier. Here, we notice the reappearance of the primitive rhythm in a different musical context. Also, the noise theme, the visual / aural theme, and the origin theme are used. At the same time, the composer attempts to separate the origin theme from the remaining themes. The section ends with the repetition of A three times, which leads to the choir. Table6. Part VIII

6) Part XIV: In terms of drama of the opera, it builds up tension as it presents the death of each character. This, consequently, tells us that we are going to reach the dramatic climax of the opera. Music in this part of the opera is simple, only featuring the Noise, Tower theme, and the unconfused theme that reveal how the drama of the opera is going to unfold. Part XIV restates the themes presented earlier. At the same time, begins to witness the death of Mirmah, Ohola, and Alex once at a time. Consequently, the death of each character coincides with the disappearance of the corresponding motive. In section 30, the

13 unconfused theme presented together with the confuse theme in an orderly, subdued manner. Sihon is accompanied by a cluster of diatonic tones, which is marked by an accent in Section 35. Table7. Part XIV

7) Part XVI: The scene is ended with the beautiful duet of Anna and Sihon who express the pain of losing their baby. The visual image and aural & the origin theme presented before reappear. The familiar theme of the gesture of the dance, the traditional Korean tune and the melody of the flute now delivers the message of truth. In Part XVI, section 37 includes Anna’s soliloquy while section 38 presents the duet with the accompaniment. Section 40 narrates what the death of the child signifies; violin solo and harmonies complete the scene. A bell is heard at the distance.

Table8. Part XVI

Effects on the MultiMulti----CCCCultureulture Structure At the outset of this paper, I defined the origin as α, namely, life, truth, and love, the messages that the composer would to deliver through this opera.

14 However, considering where she received the inspiration from, she had to speculate on how to create a universal work that can convey the messages regardless of one’s own cultural, social, and religious preferences. After all, what she would like to achieve in work is to compose a multicultural work that incorporates various musical sources found in one’s indigenous culture to the conventional from of opera. At the same time, she found that the use of multi-media in the form of visual images and acoustics is universal enough to speak to the modern audience. However, she did not hope to incorporate the multicultural aspects and the multimedia as auxiliary tools. Instead, she considered them as one of the integral parts of the opera that primarily determines the nature of the work.

At the same time, this work also allows me to ponder several questions, of which the question of genre of the work arises. Marked as a first effort to incorporate various musical as well visual aspects to the work, it does not seem to fit into our conventional knowledge of what opera is. Seen in this light, the definition of an opera in the traditional sense appears to require an expansion. This opera is not bound to a particular time and space, attempting to combine various cultures, which leads me to refer to this opera as the multi-culture-structure. In this respect, it is a revolutionary work that shows a new way for the modern opera. The composer states, “Despite the absence of a precise term, the opera, “Back to the Origin” however, propels me to carry on this type of work that I just begin to explore.”

Author Biographies

Sung Ki Min has two kinds of Ph.D. One is Mechanical Engineering and the other is economics. He is Certified Systems Engineering Professional, INCOSE and Certified Senior SE Manager, KCOSE which he founded and 3rd President. He was graduated Korean Military Academy and retired as Brigadier General. Now he is president of Institute of Systems Engineering, Inc.

Chan Hae Lee has Ph.D. in music composition. She was graduated Yonsei university and the catholic university of America. Now she is professor of. Yonsei University, visiting professor of UC, Berkley and director of Seoul contemporary Opera Co.. She got Korean composition award, two times and outstanding woman of 21th century award, ABI, USA. Her works are Opera, Musical, contemporary orchestra and the others.

15 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 14

Creating and Valuing Flexibility in Systems Architeching: Transforming Uncertainties into Opportunities Using Real Options Analysis

Ho, Wei Ling Angela Ng, Chu Ngah Defence Science & Technology Agency Defence Science & Technology Agency [email protected] [email protected]

de Neufville, Richard Massachusetts Institute of Technology Engineering Systems Division [email protected]

Copyright © 2009 by Ho, Ng and de Neufville. Published and used by APCOSE with permission.

Abstract. Flexibility is one of the strategies to ensure System-of-Systems (SoS) value delivery during its lifecycle, in light of changing external environment. This article discusses the notion of ‘Real Options Thinking’ to create flexibility and ‘Real Options Valuation’ to value flexibility when designing SoS to manage significant uncertainties. We propose a Real Options Analysis framework consisting of the dual parts of ‘Real Options Thinking’, akin to good management intuition and ‘Real Options Valuation’, akin to stochastic optimization, to promote a common “options” language, enrich vocabulary, sharpen our thinking and guide quantitative analysis when managing technical projects. Adopting real options thinking and real options valuation will increase our capacity to timely identify, create and secure both technical and managerial options through deliberate choices, cost effectively. It will also increase our tolerance of uncertainties and empower us to actively manage uncertainties to create opportunities and reduce risks. Introduction Uncertainties As Both Risks And Opportunities. In the Singapore defence context, our real-life complex systems are invariably System-of-Systems (SoS) and always operate in uncertain and dynamic environments. Typically, while Systems Architects architect SoS, Program Managers develop SoS that will operate in an uncertain and dynamic environment. The drivers of the uncertainties can be both external and internal. External factors such as emergent threats, new operational concepts, disruptive technologies and shifts in supplier industry structure are largely beyond our control. On the other hand, internal factors come from uncertainties in the program delivery and are generally well managed by current project management practices. Dealing with uncertainties arising from external factors are not well- understood today, and due to the increasing need to architect and deliver SoS in fast-changing environments, there is a need to anticipate these known-unknowns and, as far as practicable, prepare for this class of uncertainties today.

History shows that we are poor forecasters of exact trends. Overconfident forecasts like “640K ought to be enough for anyone” from Bill Gates underestimated the uncertainty in demand and technology. Designing SoS to specific trends may turn out to be costly as a single “unknown unknown” can throw off our predictions and ability to react in the future. We would be better off predicting a range of scenarios and developing the feasible solution

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 14 space of all on-demand flexible responses that we can choose to adapt as new information becomes available with time, rather than designing our SoS to forecasted trends.

Traditionally for large-scale complex systems, uncertainties are treated as risks and are therefore undesired. The approach towards uncertainties is consequently to manage them through risk mitigation. However, “Real Options Thinking” introduces a paradigm shift in looking at uncertainties as a source of opportunities as well. This paradigm shift in thinking is essentially the act of capitalizing on the opportunities embedded in uncertainties while limiting the extent of risks involved. This viewpoint is critical as most current engineering practices are conservative and may not exploit the potential upsides of uncertainty. We would be able to perform better if we can develop a dynamic strategy in anticipation of uncertainties arising from external factors. Through an SoS architecture that can evolve and adapt to a changing environment, we will increase our potential and ability to capitalize on emerging opportunities through uncertainties. Real Options Analysis is thus active uncertainty management as opposed to passive uncertainty management techniques like risk management.

Flexibility In Systems Architecting As A Strategy To Ensure SoS Value Delivery Systems architecting is an approach to design and build effective and efficient SoS. The SoS architecture "provides the structure or skeleton of the system, as well as the principles, rules and guidelines governing the system design, creation and evolution. It also provides the broad framework, system level constraints as well as relationship for the substructures and modules of the system. It determines the option available for future development." (Tan et al, 2006). In order to preserve value over its lifecycle, an SoS has to be able to handle dynamic complexity and a changing operational environment.

Flexibility is one of the key strategies to cope with environmental changes and dynamic complexity, and Real Options Analysis is one way to tangibly create and value flexibility in SoS. A working definition of “flexibility” is as follows:

Flexibility is the lifecycle property that allows an SoS to endure sets of changes with ease. It is an active and largely external approach to managing change (adapted from Moses, 2003).

An SoS is flexible if we have the freedom to make our choices during lifecycle operation, to cope with the large external changes. Flexibility can be created and valued in Systems architecting and designed using Real Options Analysis framework.

Breakthrough engineering solutions have a common thread of incorporating flexible designs to cater to wide-ranging scenarios. These flexible designs are also known as “options”, and people have been practicing “real options thinking” even though the language of options may not be as ubiquitous. A good example is the flagship Underground Ammunition Facility (UAF) project, managed by DSTA, for high-density ammunition storage. We have invested in R&D on protective infrastructure and related technologies for years to create options for MINDEF to address various defence needs. When the new challenge for creating new ammunition storage for the SAF arose in the face of tremendous pressure to free up land for national development, DSTA was ready to “exercise” the option

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 14 and embarked on the architecting and building of our first underground ammunition facility. Apart from deriving direct value from the options created through R&D, this effort also generated emergent benefits by allowing DSTA specialists to set new safety standards, including obtaining NATO’s acceptance and becoming a leader in the field.

Defining Real Options The science of options analysis began with financial options. A financial option gives one the right but not the obligation to buy the asset later at a pre-determined price. This means one can be better off during good times and have a fallback position during bad times. Should the value of the asset increase, one can profit from it. Should the value of the asset drop, one’s losses are limited to the option premium. Black and Scholes (1973) developed the theory to price financial options considering uncertainty, decision spaces and time in the equation. Option pricing has led to flexible financial structures, created a market of options transaction and reduced the volatility of the commodities.

A real option is the extension of the idea to value the flexibility of management decisions on a real or tangible asset. Asset owners need to know the price of their productive asset to determine its economic value. Since an asset can be put into creative use in multiple ways and management has the flexibility to control how much of resources to put into or remove from it in the future, - depending on the future demand - the value of the asset depends on their own possible actions. This viewpoint changed the perspective of asset valuation from historical cost valuation to prospective contingent valuation. The value is contingent on the future possible action of the management. Today, real options analysis is being accepted as a conceptual and analytical tool to support strategic decision making under uncertainty by extending existing techniques of Net Present Value (NPV) valuation. It is a bridging tool for strategic and financial decision-makers.

It is important to distinguish an “option” from our common understanding of it as an “alternative’” or a “choice”. An “option” is not another “alternative” or “choice”. Instead, it refers to our right, but not the obligation, to take an action (i.e., exercise it), at some point in time, with an upfront cost. An option is a secured choice that makes it available on-demand.

A real option is a technical or management choice that we secure today at a pre-determined cost for a pre-determined time to have the right to exercise when needed without any obligation (de Neufville, 2001).

A simple example of a real option in engineering design is the option to “expand” (the action) in the design of the bridge across the Tagus River in Lisbon, Portugal. In the 1970s, the first bridge was designed for vehicular traffic when there was no demand for commuter railway system. But the government insisted that the bridge structure be strong enough to carry rail traffic. 20 years later, there were changes in technologies and in rail demand (pre- determined time to exercise options) and the government was able to exercise its option (option to expand) and extend rail service on the bridge. There was an upfront cost for this option - the bridge had to be reinforced when it was built (pre-determined cost). However, when the environment changed in 1990s, Portugal was able to take advantage of this

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 14 opportunity to exercise the option that it had built into the technical design of the bridge 20 years earlier (Gessner and Jardim, 1998).

On face value, we may view such an example as simplistic. We caution readers to check against having any hindsight bias. We should reflect on the uncertainties and constraints that the decision-makers would have faced in the context of their time. Additionally, the decision equation will be more complicated when there are several sources of uncertainty and a few options for judgement.

Real options Types A flexible SoS must have both technical options and managerial options built into the architecture that can be exercised when needed. Both these options are classified as "Real Options" in our engineering SoS and are different from the Acquisition Options in procurement that we are familiar with.

Technical Options are options that are created in the design of the technical SoS itself. Technical Options are called Real Options “in” SoS (de Neufville, 2002) as they are embedded “in” the technical design of the SoS architecture. Identifying technical options within the SoS would require a good understanding of the SoS and a modular architecture. A classic technical option is the design of a dual operating mode. For example, straight stretches of roads can be designed and built so that we can use them as runways for aircraft. But this means we have to design the roads with some limitations for transport use while ensuring heavier load requirements and removable barriers.

Managerial Options are options that are created to manage the process of SoS development and operation. They treat technology as a “black box” and are essentially financial options taken on technical projects and SoS. Managerial options are the Real Options “on” SoS and are enabled by the technical options, contractual obligations of suppliers, financial and resource control over the process of SoS development and operations. Classic managerial options include options to defer a decision, alter the operating scale of the SoS or abandon some sub-initiatives. Back on the road-runway example, management has the option to use roads as aircraft launch pads if enabled by built-in technical options.

Real Options allow the SoS to adapt to changing scenarios over time. This flexibility increases our leverage to capitalise on upside opportunities, and limits the exposure to downside risks. Flexibility that is embedded into the technical design and management of SoS is important to improve the operational and technical effectiveness. The greater the uncertainty, the higher the value of each real option.

Both technical and managerial options come at a cost, called the option premium. The option premium can be seen in two ways: as the maximum price we should pay to have the managerial and technical flexibility or as the price of our uncertainty. Certainly we will not pay an option price that is higher than the cost of the uncertainty. When we have a bundle of options available across several review points in future, we can use a quantitative way to decide how much flexibility to incorporate into the SoS design and develop the roadmap of decisions.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 14

Real options in SoS are thus an additional technique that can be used to value the flexibility of technical choices on SoS design. It is not possible to execute many of the management choices to expand capability or switch operating modes unless the initial design had the benefit of the forethought. Securing technical options is costly and subject to much inquiry because they will not be perceived as required, unless we measure against the probable scenarios. Language of Options Thinking Technical and managerial options can be broadly classified by whether they reduce downside exposure to risks, allow status quo, or leverage on upside opportunities, depending on how uncertainties evolve. If uncertainty were deemed to have a negative impact, exercising real options like the option to downsize or mothball would reduce exposure to risk. On the other hand, if the uncertainty turned out favourably, exercising real options that increase the exposure to these opportunities would reap a greater return. Table 1 groups some of the possible options (Trigeorgis, 2001) according to this classification. In the event where more information might change the course of action, the option to remain status quo might have to be deliberately created.

Table 1 Types of Real Options

Function Options Description Option to To alter operating scale & reduce capacity by removing downsize features & resources Reduce Option to To temporarily remove from active service & put into protective risk mothball storage exposure Option to To stop and cut losses, typically when projects become terminate unprofitable Option to defer To postpone starting or initiating an investment Status Option to To maintain status quo quo continue Option to switch To move to an alternative mode of operation or design Leverage Option to To alter operating scale & expand capacity by adding features on expand & resources opportuniti Option to grow To invest in an option so it may open new options es Option to restart To restart a temporarily closed operation

Real Options Examples Real Options Thinking can be applied in different scenarios of uncertainty. In the aforementioned example of the bridge in Portugal, the uncertainty lies in the demand profile, and the Portuguese government decided to create a real option which would become valuable if the demand for a commuter railway system indeed arises. This real option gave them the readiness to switch quickly, but there is naturally an option premium involved, which includes the cost of the extra load. Table 2 provides other case examples where real options can be applied and briefly illustrates the benefit and cost impact of these considerations.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 14

Table 2 Examples of Real Options

No Uncertainty Without With option Example Benefit Cost (Option Option Type option premium) 1 Demand Single Dual use of Roads as Readiness to Operator Technical: profile may equipment equipment runways switch quickly training, cost Switch change mode of acquiring and operate both modes 2 Demand level Right sizing Spare Extra network Handle small Cost of under Technical: is volatile equipment bandwidth demand utilisation and Expand capacity changes cost of spare capacity 3 Unknown Waterfall Spiral Prototype Better risk Management Management: user development development development management cost is higher Expand, requirements lifecycle lifecycle and customer Downsize, and technical satisfaction Terminate, uncertainty Continue 4 Future Buy now Lease with Rent capacity Stop or Higher total Management: demand option to with option to continue at cost Defer purchase later purchase later favourable at lock in price price 5 Availability of Purchase Purchase with Buy x platforms Fix the price Higher total Management: new suppliers reduced set option to buy with option to buy today cost than up Continue, now or all more at lock in y more later if front buy Expand now price favourable 6 Different Customised Standardise Multi-platform Mass Design cost Management: requirements platforms base platform missiles, basic + customisation for multiple Switch from different modules interfaces users software package 7 Failure No Redundant Critical Resilience Original + Technical: likelihood of redundancy equipment as equipment for and recovery back up cost Switch operating back up 24x7 readiness equipment 8 Technical Go (high R&D to Defence R&D to Readiness to R&D cost Technical: feasibility; stakes) or investigate explore new develop into Grow Demand No-go (forgo feasibility & capability project if any gain subject feasible, else potential) knowledge exit with no further cost

Characteristics & Application of Real Options

Real Options has its pros (+) and cons (-).

+ Real Options enables staging of decisions in roadmaps. We create flexibility by building in decision review points in the future and by defining the conditions under which they should be exercised. For example, an iterative development life cycle is superior to a waterfall lifecycle when we are exploring new technical concepts. The deferred review points give management the option to scale, hold or terminate the project based on the information feedback as uncertainty unfolds. It is the flexibility to stage the decisions that potentially creates more value than a waterfall approach. Similarly a dual- purpose road is quickly used as a launch pad when an emergency need arises. Instead of formal timed reviews, the option to switch forth and backwards always exists during the lifecycle of the project.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 14

+ Options increase in value as uncertainty increases. The greater the uncertainty in a particular undertaking, the greater the need to have options, and the higher the corresponding value of the options we create. For example, the additional investment cost of extra capacity in a complex network will become viable once the uncertainty in the demand crosses a certain threshold.

- Real Options increase initial costs. Options add to the baseline configuration of the SoS and increase costs. Options are identified at the start of the project and hence, a higher initial upfront cost will need to be factored in to acquire the option at the ‘option premium’. For example, suitable civilian transportation resources are requisitioned for military purpose when needed. However, there are certain administrative and periodic trial costs to ensure that the suppliers and equipment are ready for the activation. This option premium is the price we pay for the flexibility to manage the uncertainty.

In view of the essential characteristics of Real Options, we should realise that:

• Options are derived from the SoS. An option may be a logical or physical addition to the base SoS. This concept is a useful guide when we start practising to identify options and expand our option / solution space. For instance, a missile that must have multi- platform launching capability or customised software sitting on a baseline module needs careful design of interfaces.

• Options are choices that are secured today. If we do not invest either legally or concretely to secure the choice we may need later, it is only a wish on paper. This notion is pretty clear for technical options as they have to be designed in the initial design. However, this notion can be quite subtle for management options: a procurement option is ‘purchased’ today at an option premium so that we can choose to exercise it later at a pre- agreed price.

• Option investments have to be balanced. Options are secured at a price, and we must invest in an economical portfolio that balances the benefits they give against their expense. This notion becomes apparent during front-end development when there are several sources of uncertainty with several types of option responses.

Real Options Analysis Methodology This section walks through the major steps in realising flexibility in SoS. The Seven- Step Methodology for Systems Architecting (see Figure 1) is an iterative and recursive process that moves from establishing Systems Architecting objectives to reviewing of the Architecture. While flexibility is one of the requirements in painting the big picture, ‘Real Options Thinking’ and ‘Real Options Valuation’ will enhance the generation of solution space. i.e., Step 3: Build the Big Picture and Step 4: Identify Capability Gaps.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 14

Figure 1 Real Options Methodology for Active Uncertainty Management (Tan et al, 2006)

Figure 2 is our proposed Real Options Analysis Methodology which outlines the key steps in creating and valuing the real options. The joined Red circles are the scenarios at consecutive time stages (x-axis). The uncertainty of the future naturally increases over time. The small floating circles are the potential responses (i.e., decisions, secured choices, or real options) that management and the technical team can take for each scenario. Both scenarios and responses are dependent on preceding choices and remind us of the legacy effect we may create.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 14

Figure 2 Real Options Analysis Methodology for Active Uncertainty Management

Step 1: Determine Evolving Requirements. The focus of Step 1 of the Real Options Analysis is to determine evolving requirements that can address the uncertainties in the future. This requires a hard look at multi-time stages (e.g., in Figure 2, we illustrated it with 3 time stages on the x-axis), and the probable scenarios at each time stage. We consider the technology capability today, vis-à-vis projected uncertainties (i.e., known-unknowns) coming from possible threat evolutions, key technology trends, changes in our operational concepts and operating environment, as well as stakeholders changes.

Step 2: Conceptualise Architecture. The art of Systems architecting is a subject on its own. It is likely that there are several competing architectures proposed to meet the evolving requirements. Apart from functional needs, the SoS Architect has to trade off the various strategies for coping with dynamic complexity. This develops insights into the physical and non-physical aspects of the SoS, sources of changes, and the dynamic behaviour of the SoS. The architectural descriptions should allow for qualitative understanding and quantitative inquiry for valuation of options.

Step 3: Real Options Thinking to expand solution space. Real Options present the most flexibility and value in areas of greatest uncertainties. After projecting where the biggest uncertainties lie (i.e., Step 1), we should explore the option/solution space and identify where real options can be created in the SoS. It is important to identify and clarify both managerial (i.e., Real Options “on” SoS) and technical options (i.e., Real Options “in” SoS).

Step 4: Real Options Valuation. After identifying and creating the real option space, the real options are modelled by varying the designs and consequences. The steps are outlined in Figure 3. Specifically, there are 2 models to be developed: the Ops/SoS model and Options Valuation model.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 14

i. Ops/SoS model – The Ops/SoS Model is an Operational Analysis (OA) model that either produces a suitable response for a given threat scenario and or generates the threat scenario that an SoS can handle given a specific set of constraints. It requires the operational and systems understanding in the specific engagement. At this level, a low resolution analytical or simulation model that transforms the key design inputs to measurable outputs is sufficient. We will use the Ops/SoS model to populate the response domain for the set of threat scenarios. We would then proceed to compute the transition costs between sequential responses.

ii. Options Valuation model – Given the set of threat scenarios, responses and transition costs, we proceed to define a MoE (Measure of Effectiveness) for evaluating different roadmaps and possible constraints that preclude certain roadmaps. An optimisation model that considers the uncertainty and time will facilitate any combinatorial search. A point to highlight is that the solution is a recommendation of a suitable roadmap for the entire set of threat scenarios, and not just a response to a particular threat scenario.

iii. Roadmap – In practice, a first look at the roadmap is likely to trigger further inquiry and adjustment of both choices and input data. Once refined and stable, the roadmap is a guide for action. The roadmap chosen is the ‘optimal’ roadmap with the highest MoE score that satisfies the given constraints. Note again that the roadmap is not a single path but is a set of possible paths.

iv. Tradespace – The tradespace plots the evolution of the ‘optimal’ roadmap against the cost and effectiveness. We can compare other roadmaps that are heuristically chosen to understand the differences.

v. Option value – The roadmap is derived from a probability tree. The MoE of a roadmap is an average value based on the responses for the given scenario. To have a good understanding of the variability of the MoE, we plot its probability distribution. We can then compare with any other heuristically derived roadmaps. The difference will correspond to the value of the options that we have built in.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 14

Figure 3 Real Options Valuation

Step 5: Invest in Real Options. Management is presented with the results, starting with the roadmap to respond to the set of postulated scenarios. They will have to decide which real options to invest in, in order to secure the right to respond in the future. In practice, the roadmap will trigger additional questions on new options to be built today, so that we can secure the future we want. It would likely be necessary to go back to earlier stages to re- iterate the process.

Step 6: Review Real Options. During operations, we will review if the real options invested earlier need to be exercised or deferred. If real options were built into the SoS, the management can and do have the right to exercise the options when the situation calls for it. Exercising the correct real options can only be done after re-evaluating their current benefits, and re-assessing their impacts as more information is revealed with time. The perceived value of each real option will change, (i.e., become higher or lower) over time as more information is revealed and uncertainty decreases. We have to continuously re-assess the benefits and impact of each option at major decision points, in order to select which options to exercise and whether the options should be exercised.

Conclusion We will continue to design and build large-scale and complex SoS to face a future that is uncertain. By leveraging on past programme experiences & SMEs’ input, we can broadly mapped out future likely scenarios. Uncertainty creates both risks and opportunities and our SoS has to remain flexible so that we can exercise the choices we want with future changes. We need to build real options into our technical engineering SoS that will enable such secured choices on-demand. Real options are both technical and managerial in nature. While technical options are pre-built into the SoS upfront, managerial options give us the mandate to execute the real options with time, but without obligation.

A fresh perspective of flexibility is enhanced by ‘Real Options Thinking’ and ‘Real Options Valuation’. Real Options Thinking is akin to good management intuition and gives focus by listing the canonical management choices and facilitating the identification and creation of real options. Real Options Valuation is akin to stochastic optimization, which helps us to compute the maximum economic price that we should pay for the uncertainty and recommend a roadmap that triggers further thinking. Together, we call this the “Real Options Analysis framework”. Adopting real options thinking and real options valuation will increase our capacity to timely identify, create and secure both technical and managerial options through deliberate choices, cost effectively.

We hope to promote a common “options” language, enrich vocabulary, sharpen our thinking and guide quantitative analysis when managing technical projects. Having options will always be useful especially if they spread over space and time and both management and the project management team can exercise them under the appropriate circumstances.

Flexibility is a critical consideration that will enable us to deal with uncertainty through active uncertainty management. It is active because we would be engaged in exploring what real options can be designed into the SoS. As real options are designed to be exercised in the future, we would also need to think about when these options should be exercised. It is a dynamic strategic planning exercise with the aim to secure the future,

whichever way it turns out to be and also a strategic decision-making tool for SoS architects, management and project management teams. References

Black, F. & Scholes M. 1973. The Pricing of Options and Corporate Liabilities. The Journal of Political Economy, Vol. 81, No. 3 (May - Jun., 1973), pp. 637-654. de Neufville, R. 2001. Real Options: Dealing With Uncertainty In Systems Planning And Design. 5th. International Conference on "Technology Policy and Innovation, paper for presentation, June, 2001.

———. 2002. Class notes for Engineering Systems Analysis for Design, MIT Engineering school-wide elective, Cambridge, MA.

Gessner, G. and Jardim, J. 1998. Bridge within a Bridge. Civil Engineering, October, pp. 44- 47.

Moses, J. 2003. The anatomy of large scale systems. ESD Internal Symposium, Cambridge, MA.

Tan Y.H.; Yeoh L.W.; Pang C.K.; Sim K.W. 2006. Systems Architecting for 3G SAF Transformation. DSTA Horizon 2006, Singapore.

Trigeorgis, L. 2001. "Real Options: An Overview", Chp 7 of Real Options and Investment Under Uncertainty: Classical Readings and Recent Contributions, ed. by: Schwartz, E.S. & Trigeorgis, L. MIT Press, pp. 103-134.

BIOGRAPHY

Ho Wei Ling Angela is an engineer focusing on Business Process Transformation & Enterprise Architecture for large-scale complex MINDEF/SAF Enterprise IT projects at the Defence Science & Technology Agency (DSTA). She graduated with Bachelor in Computer Science (Honours) with a focus on Human-Computer Interaction from Carnegie Mellon University, and subsequently earned a Masters of Science in Technology Policy Program (TPP), with a focus on Aeronautics and Human Factors from the Massachusetts Institute of Technology (MIT).

Ng Chu Ngah is an analyst involved in the structuring and modelling of complex systems and processes through the use of operations research, simulation techniques and combat models at the Defence Science & Technology Agency (DSTA). She received her Diplôme d'Ingénieur from Ecole Nationale des Ponts et Chaussées in 2006, as well as her Masters in Industrial and Systems Engineering and Bachelor in Engineering (First Class Honours) from the National University of Singapore in 2007 and 2006 respectively.

Dr. Richard de Neufville is Professor of Engineering Systems at the Massachusetts Institute of Technology (MIT). His research and teaching focus on inserting flexibility into the design of technological systems, and his work has been recognized through a NATO Systems Science Prize and an honorary doctorate from the Delft University of Technology. He is the author of five major texts on systems analysis in engineering, and had received along with Dr. Tao Wang the Best Paper Award at INCOSE 2006 in Orlando for his paper entitled “Identification of Real Options ‘In’ Projects”.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 10

A Real Options Framework for Defence R&D Investments in Capability Development

Choon Keat Ang Kah Hin Chai Defence Science & Technology Agency National University of Singapore Singapore Faculty of Engineering and Department of Industrial & Systems National University of Singapore Engineering Division of Engineering and Technology Block E1A, 1 Engineering Drive 2, 06-25, Management Singapore 117576 E3A #04-09, 7 Engineering Drive 1 Email: [email protected] Singapore 117574 Email: [email protected]

Copyright © 2009 by Choon Keat Ang and Kah Hin Chai. Published and used by APCOSE with permission.

Abstract. Real options theory has been proposed as a powerful framework for defence R&D investments decision but there remains a gap in explaining the modelling of defence R&D investments as real options. This paper builds on the recognition that capability development is a key objective of defence R&D investments. It further notes that strategic management literature has long recognised the role of R&D investments in creating capabilities to address future opportunities, and has proposed the framing of these capabilities real options. Building on relevant but largely independent streams of real options research in strategic and finance research, we propose a framing of defence R&D investments as real options in capability development. Valuation methods for these real options are also presented.

Introduction Strategy researchers have proposed that the real options methodology could be adopted in the evaluation of R&D investments as it more accurately models the rights but not obligation of commercialising the R&D results (e.g. Brealey and Myers (2002) and Mitchell and Hamilton (1987)). Some authors have proposed that the real options methodology could also be used in the evaluation of defence R&D investments (Rouse and Boff (2004), Ang and Chai (2007)). However, there remained a gap in the literature on the appropriate underlying asset and metric for the valuation of the embedded real options in defence R&D investments. Rouse and Boff (2004) proposed the real options embedded in a defence R&D investment can be valued from the cost savings offered by R&D relative to other acquisition approach. Thompson (2006) and Ang and Chai (2007), however, noted that small countries invest in defence R&D for strategic objectives beyond financial returns. One such non-financial return is the development of the indigenous defence technological capabilities (Ang and Chai, 2007). The defence R&D investments of Singapore, for example, have yielded both “leading-edge capabilities” for its forces and “deep and broad expertise” in its human capital (Speech, 2006). The creation of capabilities through R&D investments had been well recognised in strategic

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 10

management literature (Clarke and Pitt (1996), Helfat (1994), Cohen and Levinthal (1989)). These capabilities form platforms for future development and represent investments in future opportunities (Kogut and Kulatilaka, 1994), and could be framed as real options (Kogut and Kulatilaka, 2001). This paper builds on the strategic insight of R&D investments in capability development and relevant but largely independent streams of real options research (Tong and Reuer, 2004), and proposes a framing of defence R&D investments as real options in capability development. Promising methods for the valuation of these real options portfolio would also be presented. Literature Review Capability development from R&D investments. The multiple facets of the returns from R&D investments had been well discussed in the strategic management literature. Practitioners such as Andrew and Sirkin (2006) recognise that in addition to direct benefits such as cash, there are indirect benefits of innovation such as knowledge acquisition. Knowledge and innovation are cumulative and evolutionary (Nelson and Winter, 1982) and R&D investments enable firms to produce incremental innovation, and create technological variation or adopt technological change quickly to move with the unpredictable technological discontinuities which punctuate the technological life cycle (Tushman and Anderson, 1986). Besides the goal of developing particular technologies to meet expected market applications in the foreseeable future, the strategic objective of R&D investments is to develop firm-specific capabilities (Helfat, 1994) and the means to sustain competitive advantage for the long and uncertain term (Clarke and Pitt (1996), Cohen and Levinthal (1989), Kogut and Kulatilaka (2001)). For example, R&D capability reflects a firm’s strength in discovery and innovation, and enables it to value, assimilate and exploit new knowledge (Cohen and Levinthal, 1989). MacMillan and McGrath (2002) propose R&D investments create real options in a landscape characterized by technological and market uncertainties. Capabilities as real options. Real options are investments in physical assets, human competence, and organisational capabilities that provide the opportunity to respond to future contingent events (Kogut and Kulatilaka, 2001). For example, real options provide flexibility with a delay in a full-scaled commitment and the opportunity to switch course of action or grow the investment as knowledge is improved with resolution of the uncertainties in the landscape. Noting that capabilities are platforms that create a generic set of resources and represent investments in future opportunities, Kogut and Kulatilaka (1994, 2001) argue that capabilities could be framed as real options and propose a heuristic framing of capabilities as real options which guides the normative evaluation of exploitation and exploration. Theory of real options. Since the formal inception of real options by Myers (1973), the theory of real options has generated interest in various disciplines including strategic management, technology management, finance and systems engineering. Tong and Reuer (2007) provide an excellent summary of the literature in the strategy and finance research. Systems engineering researchers have extended the applications of real options into socio-technical systems (for example Wang and de Neuville (2004), and Kalligeros and de Neufville (2006)). Classical valuation approach. The classical valuation approach for real options is founded on financial options valuation (for example, Dixit and Pindyck (1994) and Trigeorgis (1998)). Despite the analytical appeal and history of real options, these techniques have yet to take root in

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 10

the world of practice in any broad-based fashion. The limited use of real options analysis in strategic planning may be due to the complexity of the real options approach and practical difficulties in using the methodology. These problems generally fall into three categories: finding a model whose assumptions match those of the project being analysed, determining the inputs to this model, and being able to mathematically solve the option pricing algorithm (Lander and Pinches (1998), Bowman and Moskowitz (2001)). Bowman and Moskowitz (2001) note that many of the assumptions underlying financial option valuation models do not hold in the strategic contexts of resource development and deployment, where many of the explicit features of exchange-traded options are absent. The most frequently cited classical real option valuation method is probably the Black-Scholes (Black and Scholes, 1973) model and the literature is filled with clean-cut applications of this model. While widely used in financial options valuation, there is a growing body of evidence that the assumptions underlying the standard Black-Scholes model are too simplistic when applied to pricing options on many real assets (Bruun & Bason, 2001). The model also ignores many of the complications associated with intangibles like R&D (Sudarsanam et al, 2005). The pragmatism of direct use of financial option pricing for the very different real options is also questionable, due to the difficulty in the identification and estimation of several of the option parameters needed in the model. In particular, the estimation of volatility is very difficult since the underlying investment opportunities are not traded. Historical data is also frequently unavailable due to the exploratory nature of the activities. Amram and Kulatilaka (1999), however, maintain that many of the difficulties with the Black-Scholes approach can be overcome using Monte Carlo simulation which is able to roll out thousands of possible paths of evolution of the underlying asset from the present to the option maturity or exercise date. The method can handle many aspects of real-world applications including complicated decision rules and complex relationships between the option value and the underlying asset. Simulation models can also solve path-dependent options, where the value of the options depends not only on the value of the underlying asset but also on the particular path followed by the asset. For example, investments in further customer relations initiative depend on the profitability of past customer relations. Amram and Kulatilaka (1999) also note the growth in the number of instruments traded on financial market and suggest that, increasingly, a suitable source of volatility information can be identified. Systems engineering research in valuation. Some recent systems engineering research in real options have taken a different and promising approach to the valuation of real options. An example is a stream of research at Engineering Systems Division (ESD) of the Massachusetts Institute of Technology (MIT) which extends practical tools such as Value @ Risk (VaR) - more widely used in financial analysis for the robustness of an investment portfolio - for the valuation of the flexibility embedded in the real options within a system (for example, Cardin et al (2007)). Another potentially powerful method developed at MIT ESD is metrics to evaluate the flexibility and value robustness of a system (Ross et al, 2007). These practical methods can handle non-financial returns and avoid the unrealistic assumptions of the financial methods for real options valuation. This paper builds on these works and proposes a preliminary methodology which could be further developed for the robust valuation of individual and portfolio investment in R&D.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 10 Proposed Conceptual Framework Strategic transformation process. R&D can be viewed as a process of resource transformation (Schimidt and Freeland, 1992) whereby firms can create strategic options by transforming resources into capabilities which offers strategic flexibility. These real options provide robustness in the firm’s portfolio of capital and R&D investments in a landscape characterized by technological and market uncertainties (MacMillan and McGrath, 2002). In the particular case of defence R&D investments, the specific uncertainties are the uncertainties in technology development and the fielding of the application. The portfolio of defence R&D investments would likely include investments of varying level of technological uncertainty and the level of uncertainty of fielding the application would also vary (see Table 1). The strategic objectives and expected payoff from the real options created would, hence, be varied. Table 1. Technological and scenario uncertainties in defence R&D investments Low technological uncertainty High technological uncertainty Low “Incremental innovation”. E.g. “Modular innovation”. E.g. quantum application upgrading weapon systems. The leap in weapon systems performance. uncertainty returns can be evaluated by traditional Valuation is challenged by high tech tools such as Net Present Value. uncertainty.

High “Application innovation”. E.g. fielding “Radical innovations”. E.g. R&D application existing technologies in new doctrine investments in emerging breakthrough uncertainty of operation. Valuation is difficult technology which would swing the because of uncertainty of the war. The returns of these strategic operational scenario. investments include knowledge of the firm. The valuation is difficult to evaluate due to technological and operational uncertainties. The capabilities developed in defence R&D investments can be categorised by the maturity level as follows: Systems capability. These capabilities are developed with the application of technology to address an operational requirement. An example is enhancement of operational capabilities with improvement in a weapon system. While these capabilities offer direct returns on investment for the end user, the returns may be non-financial and hence not easily measured in dollar terms. Technological capability. These are vanilla options created from investment in technological capabilities. They offer the end user technological options - the right but not obligation to further develop the technological capability into system capability. These technological options correspond to the real options in strategic positioning proposed by Mitchell and Hamilton (1988). An example is exploratory development in specific technologies. Knowledge of the firm. These are compound options created from investment in knowledge of the firm. The returns to end user may not be apparent and hence not easily quantified. The knowledge created can be considered as owning a portfolio of options, or platforms, on future developments (Kogut and Zander, 1992). An example is investment in human capital.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 10

Transformation map. Building on our recognition of the two dimensions of uncertainties in defence R&D investments and the cumulative and evolutionary nature of capability development (Nelson and Winter, 1982), we propose a transformation map for the real options embedded in defence R&D investments (see Fig 1). The large scale mission oriented projects in defence R&D aim to develop specific technologies under high appropriability and high cumulativeness (at the firm level) conditions. These lead to a Schumpeter Mark II Model of innovation regime (Breschi et al., 2000), characterized by “creative accumulation” and the importance of experience.

Figure 1. Transformation map of capabilities The transformation of an R&D investment can be seen as a development vector describing the maturing of the technology and resolution of uncertainty in the application. The real option embedded in a technology development programme evolves as the technological uncertainty decreases with technological maturity and the readiness for field transition increases with identification of application. These technologies may not even be developed and matured in a military laboratory. For example, the innovation of tanks during World War I was founded on the internal combustion engine and caterpillar track, both of which were mature technologies being used in early farm tractors. Similarly, as application uncertainty decreases with improved clarity over the fielding of the application in particular operational scenario, an R&D investment can be made to develop the requisite technology. The development of the rockets during World War II is an example wherein military application drove the technology development. Clearly, the development of the technology and concept of its application may be concurrent. In the development of aircraft for warfare, for example, the aircraft graduated from observation and reconnaissance to attack and bombing functions as the technology advanced rapidly during World War I. Implications. The proposed transformation map is more than a categorisation of uncertainties posed to a defence R&D investment. It provides a framework within which one can examine the relationship among defence R&D investments, capability development and options creation for the uncertain future. The framing of defence R&D investments as real options in capability development can help to explain the theoretical foundation for the application of real option theory to model defence R&D investment. This discussion also contributes to the literature which generally focuses on the strategic objective of long term investments as a bet for the uncertain future but omits the deliberate process of capability development to create strategic options. It helps to present the strategy of investing in a portfolio of R&D investments with different technology readiness and development of capabilities with varying level of maturity within a process of exploration and exploitation. This insight could

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 10

be useful in opening more discussion on the strategic balance of exploration and exploitation activities within a mixed portfolio consisting of long term and short term investments. Finally, this framework demonstrates that the level and nature of uncertainties and the expected returns for different R&D investments may vary. Hence, the successful pursuit of different kinds of R&D investments may require different kinds of management approach, including investment valuation. As noted by Mitchell and Hamilton (1988), capital budgeting method can be used for the valuation of an investment in product development while the highly uncertain investment in knowledge is best treated as expenses. Mitchell and Hamilton (1988) also propose that Real Options Valuation is suitable for the valuation of strategic positioning activities including applied research, exploratory development, and feasibility demonstration, with a level of uncertainty in between the two ends of the spectrum. Hence, we propose that different valuation methods are suitable for R&D investments at different stages of the resource transformation process and with different strategic objectives of exploration and exploitation. The next section of the paper will further discuss the valuation approach for the real options embedded in R&D investments. Proposed Valuation Methodology Proposed valuation approach. We have presented the strategic objectives of R&D investments in terms of real options in capability development. The real options value is the flexibility embedded in the full investment portfolio by the constituent R&D investments which provide the rights but not the obligation to build on the developed capability. Building on this principle and several practical approaches developed in previous system engineering research (Ross et al (2007), Cardin et al (2007)), we now propose a preliminary valuation approach for the real options embedded in a defence R&D investment portfolio. Our approach starts by deriving the value of the real options embedded in an individual investment. A valuation of the real options embedded in a portfolio can in turn be derived with the recognition that a portfolio is an aggregation of investments. We assume that: 1. The objective of R&D investments is to develop capabilities and create real options. The real options have value in the flexibility embedded in the individual investments and the portfolio constituting capital and R&D investments. 2. Uncertainties are defined in two dimensions: technological and application. 3. The return of a R&D investment can be described, e.g., in terms of system attributes or utility value. Within a system development programme consisting of capital and R&D investments, the embedded flexibility enables one configuration of the system to evolve into another. For example, a real option embedded in system 1 can be exercised at a cost to enable the system to evolve to system 2, while another real option can be exercised at another cost to enable the system to evolve to system 3 (see Fig 2). The value of the real option, hence, is the difference between the value of switching from one system to another and the exercise cost of the option. Metric approach. The Filtered Outdegree (Ross et al, 2007) can be used to measure the flexibility of the real options embedded within a portfolio by the number of paths a system can evolve and the cost of exercising the options. As the number of paths increases or cost of exercise decreases, the flexibility within the system increases and the real option value increases. In addition, the utility value of different portfolios can be computed and plotted. A Pareto frontier can be obtained from

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 10 each plot. By varying the range of parameters, different values and plots for the portfolios and different Pareto frontiers can be obtained. The frequency of a portfolio appearing on the range of Pareto frontier is its Pareto Trace Number (Ross et al, 2007). This metric is a measure of the value robustness of the portfolio.

Attributes or Utility

3 2

@ cost of 1 exercising option

Cost Figure 2. Evolution of a system through exercise of embedded options VaR algorithm. This approach commences with the valuation of the real options embedded in an individual investment through determining the following: 1. Conditions under which the option would be exercised. 2. Cost of exercising the option. 3. Expected return in terms of attribute or utility. 4. The value of the real option, hence, is the difference between the value of switching from one system to another and the exercise cost of the option. This function is not symmetrical and the non-negative value can be represented by C = max (0, S-K) where S is the returns which can be obtained when the option is exercised K is the cost of exercising the option

Attributes or Utility

Conditions

Figure 3. Returns with varying conditions Valuation of the real options embedded in the flexibility of a portfolio of capital and R&D investments can be determined through VaR simulation (e.g. Cardin et al, 2007) and comparing the value of the portfolios with and without the R&D investments.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 10 Conclusion This paper builds on largely independent streams of research in strategic management, technology management, finance and systems engineering and proposes a framing for defence R&D investments as real options in capability development. This framework can be a step in providing the theoretical foundation for the application of real options to model defence R&D investments, and generating more discussion on the strategic balance of exploration and exploitation activities within a mixed investment portfolio. The framework could be further developed, for example, through validation with data. The paper also presents real options valuation methods developed in the classical literature, as well as, more recent systems engineering research. The latter appears to be very promising and the paper proposes a preliminary methodology which can be further developed for the valuation of the real options embedded in individual and portfolio investment.

References Amram, M., and Kulatilaka, N. 1999. Real Options: Managing Strategic Investment in an Uncertain World. Oxford University Press. Andrew, J.P. and Sirkin, H.L. 2007. Payback: Reaping the Rewards of Innovation, Harvard Business School Press. Ang, C.K. and Chai, K.H. 2007. Real-options based approach to management of defence R&D investments: An exploratory study. In Proceedings of the Asia Pacific System Engineering Conference 2007. Black, F. and Scholes, M, 1973. The pricing of options and corporate liabilities. Journal of Political Economics 8(May-June): 637-659. Brealey, R.A. and Myers, S.C. 2002. Principles of Corporate Finance. 7 ed. Irwin/McGraw-Hill. Breschi, S., Malerba, F., Orsenigo, L., 2000. Technological regimes and Schumpeterian patterns of innovation. Economic Journal 110, 338–410. Bowman, E.H. and Moskowitz, G.T. 2001. Real Options Analysis and Strategic Decision Making. Organization Science 12(6): 772-777 Bruun, S., and Bason, P. 2001. Literature on Real Options in Venture Capital and R&D, URL: http://pages.stern.nyu.edu/~adamodar/pdfiles/eqnotes/opt1.pdf Cardin, M., Nuttall, W.J., de Neufville, R. and Dahlgren, J. 2007. Extracting Value from Uncertainty: A Methodology for Engineering Systems Design Clarke, K. and Pitt, M. 1996. R&D Initiatives and the development of strategic advantages, In R&D decisioins: Strategy, policy and innovations ed Belcher, Hassard and Procter, Routledge Cohen, W.M. and Levinthal, A. 1989. Innovation and learning: The two faces of R&D. Economic Journal 99:569–596 Dixit, A.K. and Pindyck, R.S. 1994. Investment under Uncertainty. Princeton University Press Helfat, C.E. 1994. Firm-specificity in corporate R&D. Organization Science 5:173–184 Kalligeros, K., and de Neufville, R. 2006. Real Options in Systems Design: A Methodological Compromise and its Implications. Real Options Conference, New York, June 2006.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 10

Kogut, B. and Kulatilaka, N. 1994. Options Thinking and Platform Investments: Investing and Opportunity. California Management Review 36: 52-71. ———. 2001. Capabilities as real options. Organization Science 12(6): 744-758. Kogut, B. and Zander, U. 1992. Knowledge of the firm, combinative capabilities and the replication of technology, Organization Science 3(3). Lander, D.M. and Pinches, G.E. 1998. Challenges to the practical implementation of modeling and valuing real options. Quarterly Review of Economic Finance 38:537-567. MacMillan, I.C. and McGrath, R.G. 2002. Crafting R&D portfolios. Research-Technology Management 45(5): 48-59. Mitchell, G., and Hamilton, W. 1988. Managing R&D as a strategic option, Research Technology Management 31(3): 15–22. Myers, S.C. 1977. Determinants of Corporate Borrowing, Journal of Financial Economics 5:147-175. Nelson, R.R., and Winter, S.G. 1982. An Evolutionary Theory of Economic Change. Harvard University Press. Ross, A.M., Rhodes, D.H. and Hastings, D.E. 2007. Defining System Changeability: Reconciling, Flexibility, Adaptability, Scalability, and Robustness for Maintaining System Lifecycle Value. In Proceedings of the INCOSE Symposium. Rouse, W.R., and Boff, K.R. 2004. Value-centered R&D organizations: Ten principles for characterizing, assessing, and managing value. Systems Engineering, 7(2). Schmidt. R.L. and Freeland, J.R. 1992. Recent Progress in Modeling R&D Project-Selection Processes. IEEE Transactions on Engineering Management 39(2). Speech by Singapore Minister of Defence. 2006. Opening Address by Mr Teo Chee Hean, Minister for Defence, at the Defence Technology Prize Award Ceremony on Friday 3 November 2006, at 1435 hrs in Nanyang Auditorium, NTU, URL: http://www.mindef.gov.sg/dtech/dtp/Past_DTP/2006/keynoteaddress.h tm Sudarsanam S., Sorwar G., and Marr B. 2005. A Financial Perspective on Intellectual Capital. In Perspectives on Intellectual Capital. Elsevier Butterworth-Heinemann. Thompson, M. 2006. Defence Industry in the Twenty-first Century: Implications for Smaller Countries, In International Seminar on Defence Finance and Economics, 13-15 Nov 2006 Tong, T.W., and Reuer, J.J. 2007. Introduction, In Real Options in Strategic Management, ed. Tong and Reuer, Advances in Strategic Management Vol 24, 31-66, Elsevier Ltd. Tushman, M.L., and Anderson, P. 1986. Technological Discontinuities and Organizational Environments. Administrative Science Quarterly 31(3):439-465. Trigeorgis, L. 1998. Real Options. MIT Press. Wang, T. and de Neuville, R. 2005. Real Options “in” Projects. In Proceedings of the 9th Real Options Annual International Conference.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 10 BIOGRAPHY Choon Keat Ang works with the Defence Science & Technology Agency of Singapore. He holds a Master of Civil Engineering degree from Imperial College, U.K., and a Master of Science degree in Operations Research from Columbia University, U.S.A. He is currently pursuing a PhD degree in real options and management of technology from the National University of Singapore (NUS).

Kah Hin Chai is Assistant Professor at the Department of Industrial and Systems Engineering, NUS. He teaches courses such as Cost Analysis and Engineering Economy, Quality Planning and Management, and New Product Management. His research interests include knowledge management, new service design and innovation, and new product development. He holds a PhD degree from Cambridge University in the area of International Manufacturing.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 11

Applying Systems Thinking to Solving the Problem of Vertically Integrating Taiwan’s Small and Medium Sized Enterprises

Joseph E. Kasser Willy Y.S. Peng Visiting Associate Professor Overseas Chinese Institute of Technology Temasek Defence Systems Institute 100 Chiao Guan Rd, National University of Singapore Taichung, Block E1, #05-05, Engineering Drive 2, Taiwan Singapore 117576 Email [email protected] Telephone +65 6516 4604, Fax +65 6778 9656 Cell/mobile/hand phone +65 9776 7464 Email [email protected]

Copyright © 2009 by Joseph Kasser and Willy Y.S. Peng. Published and used by APCOSE with permission. Abstract – Taiwan is a country whose industry mainly consists of Small and Medium enterprises (SME Administration, 2007). Many of these SMEs manufacture mechanical and electronic components and subassemblies. There is now a desire to move up the value chain and perform systems integration as well as the manufacture of the subassemblies of the system. However, the knowledge and skills required for systems integration are different to those needed for component and subassembly manufacture. This paper applies systems thinking to conceptualize a way in which Taiwan’s SMEs could vertically integrate1 with minimal disruption to the working of individual SMEs by creating an eastern virtual equivalent of a western large corporate systems integration house with a slight modification to the Taiwanese SME’s current way of teaming2. Keywords: Systems thinking, small and medium enterprises, vertical integration, active brainstorming, early phase systems engineering Introduction Taiwan’s Small and Medium Enterprises (SME) produce various products and services, many of them hi-tech items that are incorporated in the increasingly complex technical systems that underpin society. The SMEs are facing increasing local and global competition and have a desire to vertically integrate. Since vertically integrated organizations need knowledge, skills and competencies that SMEs may not have, the problem is to figure out a way of vertically integrating these SMEs and providing any missing knowledge and competencies. The solution

1 Vertical integration is defined as the process in which several steps in the production and/or distribution of a product or service are controlled by a single company or entity, in order to increase that company's or entity's power in the marketplace (Investorworlds.com, 2008). 2 Parts of this study are based on research which led to the award of a Doctor of Science in Engineering Management at the George Washington University, Washington DC. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 11

also has to conceptualize both a way of providing vertical integration and of producing quality products and services. This paper briefly summarises active brainstorming and then applies the technique to conceptualising a whole functional and purposeful solution to the problem of vertically integrating Taiwan’s SMEs with minimal disruption to the working of individual SMEs. The paper develops a hypothesis for a conceptual solution and ends with recommendations for a pilot scheme to test the feasibility of the conceptual solution. Active Brainstorming Active brainstorming (Kasser, 2009) is a proactive method of producing ideas relating to the problem or issue in a systemic and systematic manner. It does this in a systematic manner by examining the issue from each of nine systems thinking perspectives (STP) (Kasser and Mackley, 2008) and triggering ideas systemically and systematically by asking questions beginning with or incorporating the words “who”, “what”, “where”, “when”, “why” and “how” (Kipling, 1912). Active brainstorming uses Table 1 as a template to trigger ideas from each STP as follows. When the session begins, there will be a natural tendency to generate spontaneous ideas in an unstructured manner, particularly in a session containing newcomers to the technique. Once the initial flow of ideas stops, the facilitator starts the true active brainstorming process using the active brainstorming template at the operational perspective row of Table 1 and poses questions from column 1 beginning with or related to the word “who”. When the ideas stop flowing, the facilitator moves on to the next question in the row. At the end of the flow of ideas from the last question in a row, the facilitator moves down to the first question in the next row. Expect a question in posed one area of Table 1 to sometimes generate ideas that pertain to other areas. If no ideas come forth immediately since not all areas are pertinent to every problem, the facilitator should skip to the next question.

Table 1 Active Brainstorming Idea Triggering Worksheet STP Matrix 1 2 3 4 5 6 Who? What? Where? When? Why? How? Operational Functional Big picture Structural Generic Continuum Temporal Quantitative Scientific

Active Brainstorming the problem This part of paper applies active brainstorming to the problem of vertical integration documenting some of the questions and responses as follows: Big Picture perspective • What is the total system? The production company and its suppliers. As such, mapping the Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 11

Figure 1 The Value chain

Value Chain into the system crosses company boundaries. For example, consider the Value Chain drawn in Figure 1 across six SMEs. The functions are represented by letters and are grouped into the SME organizations (i.e. functions A and B are performed in one SME, functions J and O in a second, etc). Value is added as the products move along the chain from supplier to buyer. The initial functions produce components, the next set integrates those components into subsystems and the highest level (furthest to the right) integrates the subsystems into systems. • How many subsystems are there in a production company? Two subsystems, production and support. • What do SMEs do? They produce components and provide services. • What is driving the need for vertical integration? The desire to increase revenue share of total system production by performing systems integration. Functional perspective • What functions do SMEs perform? Production, sales, service, marketing, research, etc. Generic perspective • Who else performs the same functions? Large companies. • What is the difference between large companies and SMEs? The resources of SMEs are more limited. They lack the deep pockets and “spare” personnel of the large companies. • What is happening in elsewhere, for example in the USA? Many SMEs do the same kind of systems engineering and technical assistance (SETA), software and hardware engineering, operations support activities as do the large corporate contractors (Kasser, 1997a). • What is happening in Taiwan? A number of things. First, the Corporate Synergy Development Center (CSDC), a non-profit organization established in 1984, by the Industry Development Bureau (IDB) of Ministry of Economic Affairs (MOEA) provides technical services for SMEs including BPI, CM, e-Service, TPS, TPM, TQM, etc. Second, SMEs in Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 11

Taiwan already seem to be working together in consortia as shown by the following examples:

y Bicycle Industry - A-Team, started in 2003, led by Giant and Merida with more than 20 SMEs participating, formed a cluster in Taichung, Center part of Taiwan (Liu and Brookfield, 2007). y Tool Machine Industry - Another great example of the Taichung area industry cluster. The current success of Taiwan's machine tool industry is due to the competitive advantages achieved through flexibility, delivery, and price by the user-producer interaction of the factory satellite network system in the industry cluster in the Taichung area. The success of the diffusion of technology from bridging institutions to industry has also been an important factor (Ching-Chiang Yeh and Pao-Long Chang, 2003). y M- Team established for machine tool industry in Sept. 2007 under the IDB and The Corporate Synergy Development Center(CSDC) assistance, (Liu, 2008). y Fastener Industry - S-Team established, FPD Industry, Hand-Tool Industry, are examples of SMEs working together. • What lessons can be learned from the concept of alliances or teams of organizations? Applicable lessons learned from the USA include: y Writing about organization dynamics (Beckhard, 1969) stated that there is a considerable payoff if a new team can take a short period of time at the beginning of its life to examine collaboratively how it is going to work together, what its methods, procedures, and work relationships will be, and what the priority concerns of its members are. Then the team works more effectively, has fewer interpersonal problems, is more productive, and is more meaningful to its members. y Writing about alliances, (Drucker, 1993), pp 289-291) stated the following six issues need to be resolved for the alliance before an opportunity arises: All parties must: 1. Define their own objectives. 2. Agree on the objectives of the alliance. 3. Agree on how the alliance should be run. 4. Agree on who should manage the alliance. 5. Define the formal organizational relationship between the alliance and its own organization, including the responsibility and accountability. 6. Agree on how to resolve disagreements. Temporal perspective • Where did systems development and integration begin? It began in SMEs since the large corporations evolved from SMEs. One prominent example is the way Henry Ford vertically integrated the Ford Motor Company to ensure quality, timeliness of delivery and reliability of parts (subsystems) (Ford and Crowther, 1922). However, as time went by, vertical integration tended to produce poor quality and high inventory stocks (sometimes of poor quality parts). • How has consistent quality been guaranteed by a single entity in the past? The ISO 9000 standards were developed to ensure that consistent quality of the process and products of parts suppliers would be verified by a single impartial entity. • How is modern supply chain economics addressing the issues of quality and timelines of delivery of components? In the form of consortia of companies working together rather than Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 11

using Ford’s vertically integrated single company approach. • What about inventory costs and timeliness of delivery? Outsourcing of parts manufacturing by large companies to SMEs is common and the Just-In-Time (JIT) manufacturing/delivery concept tends to overcome issues with excessive costs of inventory and tends to ensure timeliness of delivery. Structural perspective • How can organizational structure be improved? Organizations evolved into the current hierarchical format due to issues related to “span of control”. However, flattening of the hierarchy can now be achieved by: y Management by exception (reference to be supplied) y Information technology (Rodgers, et al., 1993). Continuum perspective • What are some alternative organisational structures? The traditional systems engineering approach to organizational design allocates functions to physical elements or departments within the same organization. However, one can visualize an alternate arrangement with separate functional and organizational boundaries so that the functions can be performed by different SMEs working together as a consortium (Kasser, 1997a). This concept is similar to a temporary project organisation within a large corporation and in this context seems to be an application of the Reengineering concept (Hammer and Champy, 1993). Thus, back to the big picture perspective, the structure of a large corporation and the structure of a consortium of SMEs performing the same functions looks the same from the outside as shown in Figure 1. The difference as discussed below is that to ensure the quality of the final system/product, all the functions except Enterprise Management are performed by more than one SME in an internal (to the consortium) multiple award task ordered (MATO) contracting environment. Scientific perspective • What hypotheses could be stated for conceptualizing a way in which Taiwan’s SMEs could vertically integrate with minimal disruption to the working of individual SMEs? 1. A consortium of SMEs can form a virtual large company. 2. Systems engineering can be used to design and build the consortium. 3. Quality can be ensured by limited competition in a cooperative multiple award task ordered (MATO) contracting environment. 4. The SME consortium MATO structure offers greater benefits to the stakeholders than those available from large corporations. The hypotheses Consider each hypothesis in turn. 1. A consortium of SMEs can form a virtual large company. The virtual large company is process-based (Hammer and Champy, 1993) page 28), uses a systems approach and considers an organization as a four dimensional system (product, process, people and time) (Kasser, 1995). As such, systems engineering methodologies can be used to view, decompose and optimize the organization. The generic perspective has shown that the hypothesis has been supported by the formation of consortia in the Bicycle, Machine Tool, and Fastener Industries in Taiwan as Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 11

discussed above. 2. Systems engineering can be used to design and build the consortium. Consider the solution consortium from various STPs. The big picture perspective would be used to determine scope and range of products to be produced. The operational perspective would identify the Value Chain in as a process flow ensuring a product transfer at each functional boundary (Kasser, 1997b). In creating this consortium or designing the production system, the functional perspective of the value chain would be created first, and then the SMEs are mapped on to the value chain in the manner of Figure 2 to produce the structural perspective. In this case, the steps are:

Figure 2 Structure of Consortium

1. Identify the functions performed by the production system to produce or purchase the components, subsystems and perform systems integration. 2. Except for the Enterprise Management function, identify several SMEs who can perform each function. Thus each SME can continue to perform its core competency. 3. Note the instances where a function cannot be allocated to an SME. This is an opportunity to create a new SME or for an existing SME to grow. 4. Create the missing functions from the existing members of the consortia or invite additional SMEs to join. Distance in itself is not a factor. 3. Ensuring the quality. Quality can be ensured by always transferring products at SME boundaries. These products must be tangible items manufactured according to a specification. Quality will be assured in the contract accordance with (Crosby, 1979)’s assertion that “quality is conformance to specifications”. The products may be systems engineering process-products in the form of documents or electronic databases, or hardware and software components and assemblies. The production work is organized using a multi-contractor technical task management methodology which: y Was evaluated as "very good" by a United States Air Force Source Selection Board when incorporated in a proposal (Kasser, 1997a). y Separates the work into several tasks and produces written task work plans (TWPs). Each Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 11

TWP is a written contract between the consortium members and may be used to: y Optimize management across the organizational boundaries. y Enforce quality into the contractual structure. y Emphasizes teamwork and customer involvement. y Is tailored to the formality necessary for specific tasks of different degrees of complexity, ranging from single person tasks, to tasks requiring a large team of employees with interdisciplinary skills. y Is loosely based on a methodology used by a large contractor for eight calendar years in a task ordered environment at NASA GSFC (Kasser, 2007). y Improves on the basic methodology by adding the element of quality using the anticipatory testing paradigm (Kasser, 1995). The improvement: y Ensures work is performed in a cost-effective manner. y Maps very well into managing tasks performed in geographically distributed locations by different organizations such as in this project. y Intrinsically incorporates task management into program management. y Builds the quality into the task. y Reduces the cost of doing work. y Allows the needed staffing levels and skill mix to undergo the gradual change required to perform the planned work in an optimal manner. y Establishes baselines for the planning activity. y Monitors task and contract performance relative to the baseline plan. y Develops measurements of the effectiveness of performing the work. y Incorporates control functions that effectively deal with deviations from the baseline plan in a timely manner. The Enterprise Management Company is the holding company and provides bridging financing to the members of the consortium. When the Enterprise Management Company sets up the consortium, it arranges for a pool of teaming SMEs who agree to compete for tasks on an agreed set of criteria in a multiple-award-task-order scenario that is transparent to the customer. The single point interface between the Enterprise Management Company and the customer remains exactly the same as in the conventional approach large corporation. Work orders are transmitted from the customer to the Enterprise Management Company as in the conventional approach. The difference is in what happens to the work orders in the Enterprise Management Company. The Enterprise Management Company is a "task broker" planning, organizing, directing and measuring the work. In this scenario, the Enterprise Management Company has three functions, namely: y Breaking the work down into tasks using process architecting techniques. y Competing each task among the pool of qualified SMEs in the consortium. y Locating and qualifying additional potential consortium members. This potential inflow of new talent will tend to inhibit the consortium members from becoming complacent in their activities and keep the costs down. 4. The SME consortium MATO structure offers greater benefits to the stakeholders than those available from large corporations. This concept of a federation of SMEs performing systems integration appears to have many advantages to the SMEs, the government and the customers with few disadvantages. These Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 11 advantages and the few disadvantages are summarized as follows: Benefits to SMEs include: y No need to change core competency to join consortium. y Long term relationships with other members of the consortium help to develop future opportunities. y Consortium work can be scheduled in advance so the workload is known. y Allows for SME to work outside consortium to make use of extra capacity. y All SME need not be local, for example suppliers can be overseas. y The final Integrator in overseas contract can be a local (overseas) company. This concept has been used successfully in several industries in which components manufactured in one country have been shipped to another country and assembled/integrated in the second country by a “local” manufacturer. y Good learning and political opportunity. Teaming with an established systems integration house in an overseas country will be a good way to learn the process and create opportunities for expansion of business. Benefits to Government include: y No need to give away future tax revenue to lure large (overseas) companies. y Revenues; tax is collected from consortium (local SMEs). y Knowledge, skills and competencies remain local. y Funds support local (mainly) SMEs. Benefits to customers include: y Contractual structure ensures: y Lower probability of fraud, waste and abuse and earlier visibility of deficiencies in the system development lifecycle because the Enterprise Management Company will be aware of deficiencies at the first contractual handover point between SMEs in the value chain. y Better quality than from single large company produced system since it is in the (internal consortiums contracts) contract and reputation of SME. y Lower cost due to lower SME overheads y Increased innovation in products because SMEs are more innovative than large companies. y Probable increase in reliability due to lack of service network which provides incentive for supplier to increase reliability. The disadvantages are few, and mainly reflect the change in the manner of doing work. For example: y System integration by a consortium of SMEs is a new concept, so it will need to provide incentives to overcome resistance to change y The SMEs may have to share cost information with consortium partners who might be future competitors. However, this will depend on internal consortium financial agreements and will be different in a profit sharing or fixed price contractual environment. y Formation and ramp up will take time and probably will need government help. A pilot project to use as an example in marketing the capability to all potential stakeholders will probably be needed. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 11

Recommendations This has been an example of an early phase systems engineering feasibility study addressing a need in Taiwan. The study findings are that the concept seems to be workable and win-win. In fact Taiwanese SMEs seem to be doing something similar but have not applied it to systems integration. The logical next step up the integration path would be to form such a consortium. As a pilot project, a start could be made adapting an existing consortium such as: y The A-Team, M-Team, S-Team, Hand-Tool Industry Alliance, FPD Industry Alliance y The newly established Digital Innovative Service Industry Alliance (DISIA) by IDB of MOEA. y The innovated new Chinese life style service industry for crossing the Taiwan Strait in a single day. Many details will need to be developed. These details requiring further study include: y Types of products/systems y Contractual structure y Missing functions y Developing prototype y Leveraging on existing product lines y Potential partners The recommendation is to take the study to the next step and start with a Strength – Weakness – Opportunity – Threat (SWOT) analysis for a number of specific opportunities. Summary SMEs in Taiwan are facing the need to vertically integrate from component manufacturers to systems integrators. This paper has documented the application of active brainstorming in an example of early phase systems engineering and examined one way in which SMEs in Taiwan could form a consortium to vertically integrate in a win-win manner. Conclusion The concept spears to be feasible and win-win and should be examined in more detail with the goal of initiating a pilot project. Authors Joseph Kasser has been a practicing systems engineer for nearly 40 years and an academic for about 10 years. He is an INCOSE Fellow, the author of “A Framework for Understanding Systems Engineering” and "Applying Total Quality Management to Systems Engineering" and many INCOSE symposia papers. He is a recipient of NASA’s Manned Space Flight Awareness Award (Silver Snoopy) for quality and technical excellence for performing and directing systems engineering and other awards. He holds a Doctor of Science in Engineering Management from The George Washington University, is a Certified Manager and holds a Certified Membership of the Association for Learning Technology. He has also served as the initial president of INCOSE Australia and Region VI Representative to the INCOSE Member Board. He gave up his positions as a Deputy Director and DSTO Associate Research Professor at the Systems Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 11

Engineering and Evaluation Centre at the University of South Australia in early 2007 to move to the UK to develop the world’s first immersion course in systems engineering as a Leverhulme Visiting Professor at Cranfield University. He is currently a principal at the Right Requirement Ltd. in the UK and a Visiting Associate Professor at the National University of Singapore. Willy Y.S. Peng has been appointed as Vice President of the Overseas Chinese Institute of Technology in Taiwan. He is also the Chairman of the INCOSE Taiwan Chapter. In addition to his current jobs, he devotes most of his time to promote systems engineering education by proposing a systems engineering stream of studies in engineering college. Prior to joining the academic field, he worked in the Taiwan aerospace industry for more than 30 years and participated in two large scale systems engineering development projects from commencement to operation with internationally recognized contributions. He received both his MS and PhD degrees in Aerospace Engineering from the University of Arizona in 1969 and 1973 respectively. References Beckhard, R., Organization Development: strategies and models, Addison-Wesley Publishing Company, Inc., 1969. Ching-Chiang Yeh and Pao-Long Chang, "The Taiwan system of innovation in the tool machine industry: a case study", Journal of Engineering and Technology Management, Vol. 20 (2003), 367-380. Crosby, P. B., Quality is Free, McGraw-Hill, 1979. Drucker, P. F., Managing for the Future. The 1990s and Beyond, Truman Talley Books/Dutton, New York, 1993. Ford, H. and Crowther, S., My Life and Work, Doubleday Page & Company, New York, 1922. Hammer, M. and Champy, J., Reengineering the Corporation, HarperCollins, New York, 1993. Kasser, J. E., Applying Total Quality Management to Systems Engineering, Artech House, Boston, 1995. Kasser, J. E., "The Determination and Mitigation of Factors Inhibiting the Creation of Strategic Alliances of Small Businesses in the Government Contracting Arena," The Department of Engineering Management, The George Washington University, Washington DC, 1997a. Kasser, J. E., "Transition via Transactions: First steps in creating a customer driven organization", proceedings of First World Customer Service Congress, Tyson's Corner, VA, 1997b. Kasser, J. E., "Module 11 Reading. Program Management Methodology," in Integrated Multidisciplinary Engineering for the 21st Century, J. E. Kasser (Editor), The Right Requirement Ltd, Cranfield, 2007. Kasser, J. E., "Active Brainstorming", proceedings of the19th INCOSE International Symposium, Singapore, 2009. Kasser, J. E. and Mackley, T., "Applying systems thinking and aligning it to systems engineering", proceedings of the 18th INCOSE International Symposium, Utrecht, Holland, 2008. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 11

Kipling, J. R., "The Elephant's Child," in The Just So Stories, The Country Life Press, Garden City, NY, 1912. Liu, R.-J., "The Taiwan Machine Tool Industry System Revolution", Corporate Synergy Development Center Review, Vol. (2008), 60-65. Liu, R.-J. and Brookfield, J., "Taiwan's A-Team:Integrated Supplier Networks and Innovation in Taiwan Bicycle Industry", proceedings of Annual Meeting of the Academy of Management, Philadelphia, PA, 2007. Rodgers, T. J., Taylor, W. and Foreman, R., No-excuses Management, Doubleday, 1993. SME Administration, "White Paper on SME in Taiwan, 2007," Ministry of Economics Affairs, 2007.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 27

Systems Engineering in the Conceptual Stage: A Teaching Case Study

ZHAO1 Yang Yang Joseph E. KASSER Derek Division of Engineering and Visiting Associate Professor HITCHINS Technology Management Temasek Defence Systems Institute National University of Singapore National University of Singapore ASTEM Block E3, #04-09, Block E1, #05-05, 1 Engineering Consultant 7 Engineering Drive 1, Singapore Drive 2, Singapore 117576 Systems 117574 [email protected] Architect

Copyright © 2008 by the authors. Published and used by APCOSE with permission. Abstract. Teaching examples in systems engineering tend to fail to illustrate the conceptual problem-solving early stages in systems engineering. This paper provides an illustration by using the example of tackling the problem of providing postgraduate students with an optimal learning environment as an example of factors to be considered in the conceptual early stages of systems engineering, how the seeds of doom can be planted into a project in the early stages and how recursion and iteration are invoked in the systems engineering process. Questions and comments are provided in the text, in footnotes and in an afterword to facilitate discussion in class.

Figure 1 The HKM Framework for understanding systems engineering

1 Author’s names are given in accordance to their cultural naming order. Family names are capitalized for easy identification in accordance with Conference Country convention. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 27 Background The scope of the activities discussed in this paper are limited to the ‘identification of need’ phase in the Hitchins-Kasser-Massey framework (HKMF) (See Figure 1) for understanding systems engineering (Kasser, 2007a) particularly the activities performed in column A.2 (Kasser, et al., 2009) of the HKMF2. The paper begins by stating a need3 and then describes a systems engineering approach to tackling a problem (Hitchins, 2007) using the stated need as a starting point. The stated need is that of providing postgraduate students in systems engineering with an optimal classroom teaching and learning environment, namely a system4. The paper then demonstrates the development of conceptual alternative solutions to meeting the need by introducing and considers a sampling of the factors that affect the solutions. Sketchy concepts of the solutions are shown, discussed and a few representative risks associated with the solutions are identified. The paper then describes the development of a set of evaluation criteria and illustrates an example of decision making in a situation where the choice is unclear and more information is needed5. Having identified the conceptual solution the paper moves on to discuss the formulation of strategies and plans for implementing the solution system. The recursiveness of systems engineering is discussed at this point. However, instead of exiting the process block at this point, the need for iteration is introduced because new information is presented. The effect of iteration and

Figure 2 A systems engineering approach to problem solving (Hitchins 2007, Figure 6.2)

2 Column A addresses the conceptual solution and comprises the following sub-phases: 1. This sub-phase contains the set of activities that explore/scope the problem, leading directly to... 2. This sub-phase contains the set of activities that conceive the whole solution system (which 'emerges' from/"complements" the problem) and produces the concept of operations (CONOPS) which describes how the solution system will operate in its future environment. 3. This sub-phase contains the set of activities that design the whole solution system, identify the environment, other interacting systems, the subsystems, parts, interactions, functional architecture, physical architecture, etc., etc., - but still all of the whole. 3 Notice the need was stated not explored. Is this a common occurrence? 4 There are at least two unstated and/or implied constraints or assumptions in this situation, these being that:  The solution system is limited to the classroom environment and online distance mode and other non- traditional options are out of scope.  The content of the course meets the requirements for the knowledge components. Unstated and/or implied constraints or assumptions can have a great deal of influence on the solution system often planting the seeds of doom which lead to realisation of the wrong solution system. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 27

how to show it on the schedule and cost is then discussed. Throughout the discussion, the factors affecting the solution in the educational environment should be considered as representative, not complete. These factors are only provided for the purposes of providing an example of systems engineering in columns A.1 and A.2 of the HKMF as discussed in the paper. A systems engineering approach to tackling a problem is shown in Figure 2 (Hitchins, 2007) page 173). The approach contains the following six Tasks:  Task 1: Define the problem.  Task 2: Conceive alternative solution options.  Task 3: Identify ideal solution evaluation criteria.  Task 4: Perform trade off to find the optimum solution  Task 5: Select the preferred option  Task 6: Formulate strategies and plans to implement the preferred option. Setting the context, in this specific implementation of the approach, the activities performed in Task 1 take place in Column A.1 of the HKMF (Kasser, et al., 2009), while the activities performed in Tasks 2 – 6 take place in column A.2. Task 1 Defining the Problem Space The first task in the systems engineering approach to tackling a problem shown in Figure 2 (Hitchins, 2007) page 173) is to first define the problem space. If the problem can be expressed as a needed function, the solution becomes a system that performs the needed function namely the solution is the inverse of the problem6. In this case, the customer stated the problem7 as the need to provide postgraduate students studying systems engineering in a classroom8 with the necessary knowledge and skills components in an optimal manner and the target optimal solution is a system in the form of a classroom that provides postgraduate students studying systems engineering with the necessary knowledge and skills components in the optimal manner9. The optimal manner is defined in this situation as the best way which allows the students to ingest, retain, understand and be able to apply the required amount of knowledge in the time allocated to learning10. The customer also stated that factors affecting the solution shall include the realisation

5 Commonly known as operating under conditions of uncertainty. 6 Similarly if the problem can be stated as a function that is present but not needed, then the solution becomes a system that no longer performs the function. 7 The Task 1 activities that convert the need to a problem statement take place in column A.1 of the HKMF; hence the starting point for column A.1 should have been a statement of the need, not a statement of the problem. Does starting with a statement of the problem imply that the activities which should have been performed in column A.1 were in fact not performed? 8 This is an example of how unchallenged assumptions can lead to poor solutions. For example, challenging the assumption, one could ask is it self-evident that the solution consists of a classroom? Could it be instead, a learning laboratory, an online environment or some other alternative? Is it possible that the root problem has yet to be identified? And does ‘using a classroom’ preclude learning in an optimal manner? 9 This may also be an example in which the customer states the problem in solution language. 10 Is this definition valid? Surely, the value of a systems engineering course can be judged only by outcome, that is by the quality of the students, perhaps 3 or 4 years down the road, when they have jobs in the business and they can look back and reasonably determine what the course gave them that has proved useful. So, outcome is more valuable than output. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 27

that postgraduate students are generally employed full-time, studying part-time, and have families and other demands on their time. Consequently, the time they can allocate to their education is limited, hence the need to optimize the learning experience (Kasser, et al., 2004). At the same time, the most commonly used method for transferring knowledge11 from the instructor to the student still seems to be the lecture-based approach. The next pair of tasks in the systems engineering approach to tackling a problem may be performed in parallel. They are the tasks which conceptualize alternative solutions and identify ideal criteria for evaluating solutions. Task 2 Conceive Conceptual Solution Options A conceptual solution is one that provides a complete solution to the whole problem. Consequently, it is essential to delineate the whole problem first and then show that the conceptual solution would solve, resolve or dissolve that whole problem. Conceptualising solutions takes the form of visualising two or more conceptual solutions. The conceptual solutions can be adaptive, namely minor or incremental changes to an existing system or situation, or the conceptual solutions can be innovative (Kirton, 1994) but either way they shall solve, resolve or dissolve the whole problem. One process of identifying potential candidate solutions takes the form of asking questions about the problem and potential solutions from each of nine systems thinking perspectives using active brainstorming (Kasser, 2009a). The context for the problem/solution in this situation is a classroom12 which can be expressed as a system containing the professor, and the students. Moreover, the students can be arranged in groups or teams as shown in Figure 3. Two sets of

Figure 3 Two of the relationships in the instructor-based classroom system

11 Knowledge or understanding? What is the difference? 12 The classroom context has been insisted by the customer. In this case it demonstrates a real-world situation in which the seeds of doom (providing the wrong solution) can be planted into the project by poor very early stage systems engineering. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 27 interactions take place in the system13, the primary interaction being communications between the professor and the students, and the secondary interaction being communications between the students and the students. For simplicity the secondary actions are only shown between two student/teams, in reality there is interaction between each student/team and all the other student/teams. Three candidate conceptual solutions were identified from processing the following questions14.  Operational perspective. The following questions were posed. 1. What is the current approach? 2. How can it be improved?  Generic perspective. The following questions were posed. 1. What does the literature have to say about more effective ways of teaching and learning? 2. What lessons can be learned from other people’s teaching/learning experiences? 3. How can those lessons learned be used to solve this problem? 4. What is this problem similar to? The answers were not readily apparent so a review of the literature in the education domain was undertaken, a process of systems engineering research in the manner described by (Hall, 1962)15. This research produced two further conceptual solutions. Candidate conceptual solutions. The three candidate conceptual solutions were: A. The somewhat modified current lecture-centric classroom. B. A classroom using pedagogy based on active learning. C. A classroom environment which matches student learning styles to instructor teaching styles.16

13 It is the experience of the instructor-authors that much of the learning, where students are working in teams (syndicates), comes from interacting with other students in the team environment; essentially, the students learn from each other, and there is a social dynamic within teams. This experience matches the literature on teams in classroom environments. Empirically, different teams take on individual characters/exhibit emergent properties, indicating that they are complex subsystems in their own right. This important relationship was not considered in this case illustrating the need for domain experience as well as expertise in the systems engineering team when examining a situation. 14 Due to the implicit assumptions discussed above, two critical operational perspective questions were not posed. These questions being “what topics are being taught?” and “what topics should be taught?” the lack of these questions focused the systems engineering on the pedagogy of the classroom and incorporated the assumption that the knowledge content met an unstated implied set of requirements. This situation provides a demonstration of how easy unstated and implied assumptions can influence the development of the system in undesirable ways unknown to the participants. 15 Much of this research and the complete conversion of a lecture-based class on systems engineering to an active learning format was made possible by a grant from the Leverhulme Trust to Cranfield University in 2007. 16 Is Candidate C clearly different from the others? Candidate A might be said to match student learning styles to conventional didactic teaching styles. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 27

Consider each conceptual solution in turn. 1. The somewhat modified current lecture-centric classroom. Modifying the current approach is an obvious (intuitive?) and not necessary optimal approach which leads to a conceptual solution based on making adaptive changes. This solution is the traditional format in which the instructor is the speaker, while students are the audience. It is similar to a conference presentation session but lasts longer. It is the most familiar teaching style for students since17 they have experienced that format in the classroom since they began their education in the first grade. Modifications could start by putting the emphasis on deep learning (Biggs, 1999) and taking into account the effect of the ‘attention span’ of the students. Students seem to have limited attention spans (Mills, 1953) page 32). They tend to be more attentive at the start of a lecture, as shown in Figure 4 so the effectiveness of the lecturing decreases over time18. This means that a break should be taken after an hour or so. If one is not taken, after an hour and half, there is a good probability that at least one person will need to answer the call of nature. If they are counting down the seconds till the break because they do not wish to disturb the class, they are not learning. (Mills, 1953) also discusses the way time should be allocated in the classroom based on providing training during World War II. Mills presented the data shown in Figure 5 (Mills, 1953) page 39). Mills’ data would be useful in modifying the pedagogy of the lecture-based conceptual alternative. 2. A classroom using pedagogy based on active learning. This conceptual candidate solution is based on active learning which engages the students in the learning process rather than allow them to passively receive information from the instructor. Active learning is more than the instruction to “read, learn and inwardly digest” given out to one of the authors in Dame Alice Owens Grammar School for Boys by a teacher whose real name is long forgotten but whose nickname was “Cheyenne”. Active learning has its roots in the often quoted learning pyramid developed in the 1960’s at the National Training Laboratories, Bethel, Maine (Lowery, 2002), which list the effectiveness of

Figure 4 Attention span (Mills, 1953) Figure 5 Classroom time (Mills, 1953)

17 Students, note this example of critical thinking in the supporting part of the statement following the word ‘since’. 18 Conference sessions may have been originally limited to 20-40 minutes for this reason. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 27

seven teaching methods and in the earlier Dale cone of experience19 (Dale, 1954) page 43). The meaning of the term ‘active learning’ covers a broad spectrum of team work exercises ranging from 20-minute problem solving exercises to the way in which postgraduate business schools tend to work, i.e., where a lecturer introduces a subject, sets the class a problem based in the subject, and the class then spits into their teams to work on the problem20, perhaps for a week, finally presenting their solutions in competition at the end of the week21. This conceptual candidate uses active learning approach set in the middle of the spectrum and is also the approach used in the world’s first immersion course in systems engineering which ran twice in both Cranfield University in the UK and the National University of Singapore (NUS) in 2007 and 2008 (Kasser, 2007a). According to (Kasser, 2007a) the immersion course format produced better results in the university classroom than previous lecture-based classes did when the course was delivered in the intensive block mode format22. 3. Match student learning styles to instructor teaching styles. According to (Felder and Silverman, 1988) “Students learn in many ways— by seeing and hearing; reflecting and acting; reasoning logically and intuitively; memorizing and visualizing and drawing analogies and building mathematical models; steadily and in fits and starts. Teaching methods also vary. Some instructors lecture, others demonstrate or discuss; some focus on principles and others on applications; some emphasize memory and others understanding. How much a given student learns in a class is governed in part by that student’s native ability and prior preparation but also by the compatibility of his or her learning style and the instructor’s teaching style. Mismatches exist between common learning styles of engineering students and traditional teaching styles of engineering professors. In consequence, students become bored and inattentive in class, do poorly on tests, get discouraged about the courses, the curriculum, and themselves, and in some cases change to other curricula or drop out of school.” This conceptual candidate looks promising. However there are many problems related to the matching between learning and teaching (Dunn and Dunn, 1979). Among others, the following questions should be answered23.  What are the problems in matching teaching and learning styles?  How to design a matching teaching& learning system?  Should the matching be done before or after students select a course?  What should be the speed of the match, gradual or sudden? Each of these questions must be answered. Research and various types of analysis

19 Which does not have any numbers associated with the cone. 20 Creating the knowledge and applying it to solve the problem 21 In the author’s experience when teaching systems engineers in a corporate environment, competition between teams was a valuable spur. Offering trivial prizes was a great help, as was getting the students themselves to decide after each presentation event which team was the best - and why. 22 The class ran for four consecutive days in the first three iterations and five consecutive days in the last iteration. Students then had up to 60 days to complete the assignments. Communications with the instructor during those 60 days was encouraged. 23 These questions are broad and may require substantial analysis to determine the pertinent parts of the findings of research performed in generating the answers to the questions. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 27

and modelling/simulation tools may have to be employed. If the questions cannot be completely answered the elements of the solution they influence must be monitored. Notice that at this time the descriptions of the three conceptual solutions are just sketchy concepts of operation at different levels of detail with few if any details as to how the solutions would actually function. At this point in time a milestone review takes place. Once the conceptual alternatives are accepted as being feasible, each conceptual solution would then be developed further developing an understanding of the relationships between the instructors, students and teams and how those relationships are affected by various parameters to produce a concept of operations (CONOPS) which describes in greater detail how the conceptual solution system will operate in its future environment. Models and simulations would be developed and experiments would be carried out24. However, should a ‘show stopper’ which indicates that the solution is not feasible show up at any time, work on that solution should immediately stop. Any further effort pursuing an infeasible solution is a waste of resources. Task 3 Identify Ideal Solution Evaluation Criteria An initial but incomplete set of solution evaluation criteria can often be extracted from personal experience and the literature during the same literature search performed to generate ideas for the conceptual solutions25. However in this case the literature review26 on systems engineering education and curriculum design (e.g. (Thissen, 1997; Asbjornsen and Hamann, 2000; Brown and Scherer, 2000; Sage, 2000; van Peppen and van der Ploeg, 2000; Jain and Verma, 2007)) found that publications tended to focus on the body of knowledge for systems engineering and tended to ignore pedagogical issues, namely how systems engineering classes should be taught (Kasser, et al., 2008). (Valerdi, et al., 2009) believe that it is plausible that engineering students may prefer different learning styles depending on the content and the kind of assessment expectations which are placed upon them with respect to the abilities that they will be able to demonstrate as a result of the their study. Some evaluation criteria can be derived from student experience in a postgraduate class on systems engineering in early 2009 which employed three instructors, one after the other, teaching different topics at different levels of abstraction using different teaching styles. Student perceptions of the amount they learnt from each instructor and the differences between the instructors, the types of knowledge and the topics taught were examined and analysed to determine if the results of the analysis could provide evaluation criteria as described herein27. The variables/parameters

24 These activities are really beneficial to gaining an understanding of the nature of both the problem and the solution and are a tried and tested method of developing understanding in other applied sciences - physics in particular, plus electronics, conflict management, psychology, ecology, business management, etc. 25 This section of the paper represents the development of evaluation criteria and the presentation of information to help making the decision as to which of the alternative conceptual solutions to choose. 26 Students – take note of the example of one way to cite a literature review performed by a third party and summarize the findings. 27 But should students be the only source? Is it reasonable to judge relative merits of courses and instructors on the basis of student perceptions? Are students able to judge how much they have learned (and understood?), and are they able to separate their judgement from their emotions? Is this situation is akin to design departments making decisions on what they think the customer would want without actually asking Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 27

in the course included:  Types of knowledge.  Level of abstraction of the course content associated with the topics taught.  Instructor teaching styles.  Topics – each instructor provided a different part of the knowledge component28.  Student learning styles. Types of knowledge. (Woolfolk, 1998) described the following three types of knowledge: 1. Declarative knowledge. Knowledge that can be declared in some manner. It is “knowing that” something is the case. Describing a process is declarative knowledge. 2. Procedural knowledge. Knowing how to do something. It must be demonstrated; performing the process demonstrates procedural knowledge. 3. Conditional knowledge. Knowing when and why to apply the declarative and procedural knowledge. Prof A provided knowledge using lectures, readings and problem-based active learning. Prof A’s teaching style emphasizes conditional knowledge, rather than declarative and procedural knowledge. Prof A’s style affects the students in three ways. It: 1. Improves the thinking skills of the students. Prof A provides the outlines and abstracts or overviews of knowledge, and asks open-end questions expecting the students to find the answers and explanations by themselves or in groups. Prof A watches student teams at work and gently nudges them along the path of learning rather than leading the way. 2. Builds team-working spirit, the different group exercises following the introductory lecture are designed for ‘learning by doing’ in every class. 3. Enriches their experience in receiving the knowledge. Prof A uses multi-media (audio, video and reading materials) as additional knowledge sources for students. Examples included:  Using a virtual guest speaker. A 30 minute video on “systems” by Prof Derek Hitchins on systems engineering was downloaded from his web site (Hitchins, 2009) and played to the students. After the video had ended Prof Hitchins was contacted via Skype29 and Prof A facilitated a short question-answer and discussion session.  Having to be at an overseas International Workshop, instead of missing an evening, the short lecture was recorded and played to the students by a colleague in the classroom. Prof A then checked in to the class using Skype30 and by the judicious position of the camera on the laptop in the classroom by the colleague was able to view the students working on their

the customer. Is it also similar to a group only using items they have invented or developed in-house or have direct experience? 28 Once again the correctness of the knowledge was assumed. 29 By prior arrangement. 30 At 0330 his local time! Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 27

class exercise, view their presentations and make constructive comments.  Setting a pre-class exercise in which the students were required to download Tiger Pro, an educational requirements tool containing some artificial intelligence that can tell the students if the requirements they write are bad from the testing perspective (Kasser, 2007b). The students downloaded the tool, did the exercise individually before class and submitted an individual presentation on what they had done. Prof A subsequently compiled a summary presentation containing the student- written requirements (anonymously) which showed how and why student written requirements were good and bad.  Using the video “Pentagon Wars” (Benjamin, 1998) as a case study. The students were given a set of questions before class, watched the video in class and then answered the questions post-class in their teams. While the students did not seem to realize it, Prof A noted that lessons the students learnt from the video were indeed insightful. Prof B provides the students with the traditional and familiar lecture using PowerPoint presentation graphics. Prof B teaches the declarative knowledge and demonstrates procedural knowledge in the daily examples within the lecture. All the key information (e.g. concepts, methodology, examples, etc) are written clearly. Prof B even enunciates ‘word by word’ the content of the slides. This traditional method has been widely accepted by the students and makes most of them feel comfortable. Prof C teaches procedural knowledge in class. Prof C delivers knowledge using a combination of the traditional lecture followed by immediate group work. Prof C gets involved in the group work and personally interacts with the students and the groups as a consultant and facilitator. At the end of the exercises depending on the available time, the groups make presentations and share learning. In summary, there is a difference in the type of knowledge taught by the instructors. Prof A focuses on delivering conditional knowledge, while Prof B and C focus on declarative and procedural knowledge, which make students feel more comfortable (Kasser, 2009b). Some students can’t get used to the problem based learning method in Prof A’s class because the highly abstract lecture makes them feel unclear about what they have learned. On the other hand, Prof B and Prof C deliver the typical lecture-based class with concrete information in the slides which helps the student understand the basic concepts. Moreover, Prof A and Prof C both employ some forms of active learning. Besides those methods, Prof A’s class also involves more up-front investment in teaching resources and methods, such as identifying and creating readings, videos, etc. Topics and level of abstraction of course content. The topics and degree of abstraction of the course content was different as shown in Table 1. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 27

Table 1 Coursework content assessment Coursework Topics Level of Abstraction Prof A Critical thinking High Problem solving Context of system engineering System design life cycle Requirements engineering Prof B Risk Management Low System Real Options Prof C Business Process Reengineering (BPR) concepts Medium Process mapping and analysis Process validation BPR practice Teaching styles. The teaching styles and type of content was different for each instructor. The Learning Pyramid values for the degree of retention of information of the student after two weeks for each of the teaching methods (Lowery, 2002) and the approximate percentage of time allocated by the three instructors to each of the teaching methods is shown in Table 2. Two of the three instructors performed a self assessment of their teaching styles using an online Grasha-Riechmann (Grasha, 1996) pages 127 and 128) test31 in May 2009. The results are shown in Table 332.

Table 2 Approximate percentage of time each instructor spent in a teaching method Teaching Method Learning Pyramid Prof A Prof B Prof C Lecture 5% 30% 50% 50% Reading 10% 15% -- -- Audio visual 20% 25%1 -- -- Demonstration 30% -- 50% -- Discussion group 50% 30%2 -- 50%2 Practice by doing 75% 30%2 -- -- Teaching others/immediate use 90% -- -- 50%2 Notes 1. One class session used the movie ‘Pentagon Wars’ (Benjamin, 1998) as the basis for a case study. 2. The activities in the two rows in the column happened simultaneously.

31 Available at http://www.longleaf.net/teachingstyle.html in May 2009. 32 Further research will have to be done to determine the significance of the differences if the information is deemed pertinent to providing the solution. This is illustrative of a situation in which analysis data is incomplete. In such instances if the solution system may be affected by the incomplete information, then the missing information become ‘risks’ and shall be managed appropriately. The self-assessment was done because Web site showed up on a search and the test was simple and fast. This situation illustrates that while systems engineers measure and perform analysis it is very easy for analysis-paralysis to set in. For example, questions such as “did the test provide any useful data?” and even “why are we measuring this Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 12 of 27

Table 3 Grasha-Riechmann Instructor Self-Assessment Results Prof A Prof B Prof C Expert 3.5 Moderate No data 4.375 High Formal authority 4.25 High No data 3.625 High Personal model 4.25 High No data 3.627 High Facilitator 4.25 High No data 3.75 Moderate Delegator 3.87 High No data 3.5 High

Student learning styles. There were about 30 students in the class and using a tailored version of grounded theory (Glaser, 1992), eight students were interviewed about the class and their learning styles using face-to-face and telephone discussions. Each interview lasted about 30 minutes. The student responses were grouped into three types according to (Myers and Myers, 1980)’s personality research.  Students in Type 1 are introvert thinkers. They prefer a quiet environment for learning and listening rather than talking and interacting in class. They make decisions and work directly with data, rather than with feelings, emotions and personal values. They are objective decision makers, who like to get opinions based on established facts, known procedures and linear presentations. This type of students tends to have stronger skills in memorizing details rather than in understanding abstract picture. They prefer concrete language and working directly with data. They tend to reserve judgement until all the data has been processed.  Students in Type 2 are more likely to make decisions based on emotions, personal values or vague intuitions. They value group harmony and feel less comfortable with personal conflicts. These students tend to have stronger skills in memorizing details rather than in understanding the whole picture.  Students in Type 3 feel more comfortable interacting with others and like talking aloud in public. They believe data and evidence, but most of the time make immediate decisions and drew premature conclusions based on initial inputs. They feel comfortable with accepting abstract knowledge and get the big picture of things first. They then look inside at the internal components, items such as the connections between seemingly random sets of data, and fill in the details later. Student comments on the instructor’s teaching styles, by Type, included:  Type 1: I felt puzzled when I attend the Prof A’s class. There were too many class activities that made learning experience complex. My team members and I always feel stressful and find it hard to enjoy the class. The content of Prof’s B’s class was also not easy, but I am quite familiar with this traditional teaching method. So it is not a problem for me to grasp the knowledge. Prof C’s class made us feel easy to catch up and the number of activities is neither too much nor too little, which even inspire our interest in learning more after class.  Type 2: Prof A’s teaching style was quite new for most of us. We didn’t have enough

characteristic?” should be asked and answered. Analysis shall only be done if pertinent to conceptualising the solution, not because the data is available. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 13 of 27

psychological preparation and needed time to adapt to the teaching method. Though the organization of the teaching style is simple in Prof B’s class, the demonstration and lecture notes have enough detail for us to understand the knowledge. Moreover, the active individual presentation skill kind of balances the boring teaching method. Prof C’s class is fun. I like the immediate practice in class, which make me feel effective learning and inspires my interest.  Type 3: Prof A’s lecture is at a higher level abstract for the topics, which make it hard for most of us to grasp them in the short time. But after the module, I felt I learned more and my thinking ability improved in Prof A’s session, though it is still hard for me to connect it with our daily experience. We are used to Prof B’s way of teaching. Though it is a little boring, I feel it doesn’t depress our learning effect. What’s more, his active personal presentation skills kind of balance the boring teaching method. Prof C’s class makes us feel that it is easy to understand the knowledge through the immediate practice. Moreover, it makes everyone perform actively, because there are more chances to consult with Prof C personally in class during the team project.

Table 4 Summary of evaluation of alternative solutions Conceptual Solution 1 Solution 2 Solution 3 Solution The somewhat A classroom using A classroom modified current pedagogy based on environment which lecture-centric active learning. matches student classroom. learning styles to instructor teaching styles. Criteria Teaching styles Does not allow Multiple styles but Matched to student much variation. not matched. learning styles. Types of knowledge All All All Topics All All All Degree of abstraction Suitable Suitable Suitable of the course content Student learning Does not take Variation in Takes student styles student learning activities seems to learning styles into styles into account. allow for different account. learning styles at different times in the class.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 14 of 27

In this classroom example, from the random sample33, the majority of the students are introverts and thinking students, perhaps because of their prior engineering background. But the majority also agreed that classroom interaction and being an extrovert are also good for learning. They hoped they could be more extroverted and sociable in the light of their perceptions of the types of students in the business school. These surveyed students would like to become managers in future, managers who can perform decision making and risk management at the business level, rather than remaining as a person who can only deal with data. As the content of their degree program is positioned between engineering and business, and given their prior major engineering background, their preference for subjective and objective decision making is relatively equal.

Task 4 Trade off to Find Optimum Solution The analysis identified variables and data that could have an effect on the solution but the results of the analysis are inconclusive34. When the results are summarised as shown in Table 4 the data does not appear to be useful and there is no data upon which to make an objective decision as to which of the conceptual alternative solutions to pick35. The evaluation criteria in this case had been determined using student provided data. But are the students a good source of evaluation criteria? There are other stakeholders – instructors, employers and the academic institution (Kasser, et al., 2008). Students can only evaluate that the way in which they were taught, they cannot evaluate that they were taught what they need to know (at least not immediately after the class ends)36. Other evaluation criteria need to be identified. Looking out of the box by posing the generic perspective question “What is this problem similar to” one answer is a digital radio communications system where the ‘ability to apply the knowledge in various situations’ is the message, the instructor is the transmitter, the student is the receiver and the amount of received signal represents the learning. Maximising the received signal requires that the transmitter and receiver be on the same frequency, use the same modulation, compatible data rates and the message is transferred in an environment with minimal interference. If this analogy holds then the selected solution should be one which matches instructor teaching styles to student learning styles unless a thorough search of the education literature and the opinions of cognizant personnel in the education domain would confirm that in the last 20 years or

33 It needs to be mentioned that the survey results may be biased and limited. This is because people tend to complain during evaluations and sometimes blame others subjectively rather than cite good points. In addition, students get used to relying on the teacher actually teaching in class, and not having to do it themselves (DIY) or self-learn. Moreover, students are reluctant to change their learning styles. Resistance to change is an important element that has to be taken into account when introducing change into any context. 34 How much of the data is pertinent and how much is not? This is where experience is used to separate signals (pertinent data) from noise (data that is not pertinent). 35 Had there been domain experts in the systems engineering team the results of the analysis might have been different. This result is meant to illustrate the need to have problem domain expertise and experience during the systems engineering problem solving activities. 36 And will not pick up or question the implied assumption that the knowledge component is correct and complete. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 15 of 27

so, research has shown that matching teaching The Five Types of Systems and learning styles makes no significant Engineers difference in the effectiveness of learning  Type I. This type is an “apprentice systems engineering. Or should it? who can be told “how” to implement The accuracy of the generic perspective the solution and can then implement it. analogy is critical to the success of the project;  Type II. This type is the most common in this analogy the message is akin to the type of systems engineer. Type II’s ‘ability to apply the knowledge in various have the ability to use the systems situations’. This analogy would drive the engineering process to figure out how pedagogy towards produce Type V systems to implement a physical solution once engineers (Kasser, et al., 2009) – see sidebar. told what conceptual solution to Had the analogy stated the message as just implement. being akin to the ‘knowledge’, the analogy  Type III. Once given a statement of the problem, this type has the would tend to drive the pedagogy towards necessary know‐how to conceptualize producing Type II systems engineers which the solution and to plan the seems to be common practice (Kasser, et al., implementation of the solution. 2009) since much of systems engineering is  Type IV. This type has the ability to now taught as declarative and procedural examine the situation and define the knowledge as defined by (Woolfolk, 1998). To problem. be fair, this focus on declarative and procedural  Type V. This type combines the knowledge is not unique to systems engineering abilities of the Types III and IV, (Microsoft, 2008). For example, Peter Drucker namely has the ability to examine the situation, define the problem, wrote “Throughout management science--in the conceptualise the solution and plan literature as well as in the work in progress--the the implementation of the physical emphasis is on techniques rather than solution. principles, on mechanics rather than decisions, on tools rather than on results, and, above all, on efficiency of the part rather than on performance of the whole."(Drucker, 1973) page 509)37. Notice that the conceptual alternative solutions have been developed without regard to cost and implementation constraints. At this stage in the systems engineering problem solving paradigm these constraints should be two of the evaluation criteria. The only time the constraints are to be considered in the development of a conceptual solution is if they become a show stopper and indicate that the solution is not feasible as discussed above.

Task 5 Select Preferred Option The next task in the systems engineering problem solving paradigm is to select the preferred option. While the findings of the analysis are not firm, the preferred approach from the literature and the digital radio analogy is to choose an option that incorporates matching teaching to learning styles. However, the amount of work to implement the solution is unknown since the cited work was published in 1988 and systems engineering

37 Today’s academic institutions seem to be producing Type II systems engineers and managers (engineer leaders); but they should be producing or at least identifying personnel with Type V characteristics by teaching conditional knowledge. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 16 of 27

classes are still in the main lecture-based. On the other hand, the change from the lecture- based style to the matching teaching and learning styles constitutes a paradigm shift and has experienced resistance to change. Unwillingness to unlearn something is a major cause of resistance to change. (Drucker, 1985) stated that a paradigm shift in management takes about 25 years, namely the time it takes for the ‘unwilling to unlearn’ proponents of the old paradigm to retire. (Kuhn, 1970) also mentions the generational delay in making a paradigm change. A simple calculation on the back of an envelope shows that 2009 – 1988 = 21. There is hope that perhaps the time to make the paradigm change is approaching. However, hoping for a solution does not guarantee success. The reasons for the resistance to the change need to be investigated, and become a prime candidate for risk determination and mitigation. Other risks to be monitored and mitigated might include those associated with matching teaching and learning styles mentioned above and issues arising from the following questions.  Does the type of content affect the desired learning style?  Is an individual learning style fixed or does it vary in some manner? A fourth option? There may even be a fourth option in situations where the alternatives have been developed by different teams. Each team will generally have different degrees of expertise in different domains and produce conceptual solutions containing useful ideas. It is then likely that a fourth option could be put together based on integrating concepts from the first set of alternatives. Should this situation show up, then the process must iterate back to the start of the phase and develop the fourth conceptual solution38. For example, Option 1 was the somewhat modified current lecture-based classroom. It was a generic conceptual solution that would teach students in the same way. The detailed conceptual design based on further research identified eight factors for a growing and expanding teaching system (Fitzgerald, 2005). The following four factors were incorporated into the instructor’s notes in the conceptual design for Option 1:  Plan the competencies you want students to develop.  Prepare the learning options and choices. This can be a school-wide program in which students learn about different learning styles and talent choices and how to use their strengths.  Give students a choice of teaching styles which means provide different ways for students to receive information in the manner of Prof A. Examples would be auditory, visual, somatic and reflective experiences.  Check and adjust. For example, if formative assessment shows that teaching technique X did not work for a student, you might try technique Y as an “adjustment”. The other four factors39 listed by Fitzgerald were considered out of scope in this

38 Unless the problem is in a well-understood domain, the schedule should always be planned to contain at least one iteration cycle. What would the number of iteration cycles depend on? 39 The other four factors (Fitzgerald, 2005) that were not considered in this case study, are listed herein for further discussion:  Begin with motivation or connection (to real life) activities. Show students how the lesson(s) will give them power and how you will help them (anchor). Give them a unique, question-generating introduction (hook) to the lesson topics. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 17 of 27

classroom system because those factors were deemed to affect the meta-system namely the doctrine, institutional characteristics, organizational goals, resources, etc in the academic institution rather than in a single classroom40. Research also discovered that other factors such as teacher beliefs, teacher understanding of their roles, the syllabus time available, the textbook and the topic, preparation and the available resources, and the language proficiency of the students also need to be considered in the design of a classroom (Carless, 2003)41. The goal of the solution is to optimize the teaching system. This means that the pertinent factors should be incorporated into the instructor’s notes in the design of the selected conceptual alternative even if they were not part of the original conceptual solution42.

Task 6 Formulate Strategies and Plans to Implement The next task in the process is to formulate strategies and plans for the implementation of the preferred solution. The preferred conceptual option is to match teaching and learning styles; the findings from the analysis show that the students have to be prepared for any teaching style that is not lecture-based. As such some sort of training will have to be provided. The new problem becomes how to provide both the course and the training without extending the classroom time? This is where the recursive nature of systems engineering is illustrated, because the approach to providing a solution to this new problem is the exact same six tasks shown in Figure 2 even though they all take place inside this Task as sub tasks. In this situation conceptual solutions might include:  Providing training to instructors teaching in the programme.  Providing training to the students at the start of every class in the degree programme.  Providing training to the students at the start of the degree programme.  Don’t provide training to the students but identify individual student learning styles and stream the students in classes taught by different instructors using different teaching styles (either for the entire degree programme or individual classes) which match the learning styles of the students.

 Provide different tasks so that students can use talent or intelligence preferences and develop other talent strengths to demonstrate learning. Here you have tasks that let them "construct meaning," the true measure of learning. This is the area of multiple intelligences.  Promote mastery with some different re-teaching where necessary. Some of this might come after formative assessment The goal is student competence, i.e. not "20% missed is good enough!"  Celebrate success. Document competencies achieved or not achieved so that the next teacher can follow-up properly. Remember that congratulations help students take pride in learning. 40 Factors that affect the meta-system need to be addressed in the context of the meta-system configuration control and not just noted in the single system and ignored/forgotten! 41 However, this research was performed in a primary school in Hong Kong and the relevance of factors affecting primary school teaching in that culture to those factors affecting postgraduate systems engineering teaching in this culture needs to be determined and if, and only if, applicable, incorporated into the preferred conceptual solution. This anecdote illustrates a serious issue. Lessons learned from one context shall not be incorporated into another context without a full understanding of the context from which those lessons were learnt and how much of that context is applicable on the current project. 42 Similarly in any project, serious consideration should be given to incorporating factors contributing to the success of one conceptual alternative solution into the preferred solution if those factors are solution independent. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 18 of 27

The feasibility of each of these alternatives would have to be determined, evaluation criteria established, a solution chosen and the appropriate CONOPS and implementation plans developed. Given that there is a high degree of uncertainty the solution classroom system should be implemented in phases using an evolutionary approach such as (Kasser, 2002). The plans for the implementation of the solution system may be documented in the Systems Engineering Management Plan (SEMP).

Iteration required Hold everything! New information has come to light43. Further research found a report addressing the issue of the relationship between instructor’s learning style, student style and the impact that the degree of instructor commitment to a particular mode may have on student achievement in students enrolled in freshman English (Davis, et al., 1988). The method used to assess student learning styles was based on (Kolb, 1984). The results in the report stated that there was no significant difference in course grade point averages among students who were matched or partially matched with, or held opposite styles to, their instructors. The Davis report provided several reasons for the inconclusive results and recommended further research. These findings need further research. However, if the findings are valid then a new alternative may become feasible. As mentioned above, the majority of students in the sample are introverts. This situation is supported by the newly discovered statement by (McClure, 2004) who when reviewing (Laney, 2002) began the review with “Are you an introvert? Only a quarter of the general population is, but more than half of engineers are”. Another set of conceptual alternative solutions for classes teaching systems engineering could be based on (1) verifying a hypothesis that systems engineers, as a subset of engineers, tend to be introverts and then (2) creating a classroom teaching and learning system based on the learning styles of introverts as the norm rather than on matching teaching and learning styles. The literature on learning styles contains a number of different ways of expressing and evaluating learning styles including:  The VARK (Visual, Aural/Auditory, Read/Write, and Kinesthetic) learning style

Table 5 ILS Learning and teaching styles Learning Style Teaching Style Sensory, intuitive-perception Concrete, abstract-content Visual, auditory-input Visual, verbal-presentation Inductive, deductive-organization Inductive, deductive-organization Active, reflective-processing Active, passive-student participation Sequential, global- understanding Sequential, global-perspective

43 This paragraph demonstrates that changes in the state of the art as well as the customer can provide new information or describe a new need anytime in the development of the solution system and sometimes that new information means that the work already done is no longer valid and the process has to start again, from the beginning of the appropriate earlier stage. In practice this restart delays the completion of the project by the amount of time taken to get back to the point at which the iteration began (schedule delay) Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 19 of 27

instrument which divides learning styles in response to the input forms of ‘visual’, ‘aural/auditory’, ‘read/write’ and ‘kinesthetic’ forms (Fleming and Mills, 1992).  The Grasha-Reichmann model developed in the early 1970s and used for more than two decades to identify the preferences learners have for interacting with peers and the Figure 6 Representation of Figure 1 in instructors in classroom settings Gantt chart Format (Grasha, 1996) pages 127 and 128).  The Index of Learning Styles (ILS), (Felder and Soloman, 2008), a model which classifies instructional methods according to how well they match the teaching and learning styles shown in Table 5. The documents would be reviewed and data extracted and correlated to the needs of introverts and a new set of conceptual alternative solutions conceived. These conceptual solutions could even provide a lower cost more effective solution than those already identified and minimise or even eliminate the need for training students and

Figure 7 Gantt chart representation of schedule delay due to Iteration of full sequence of tasks

and incurs costs due to the resources expended in the unplanned iteration (cost escalations). Good project planning should allow for a number of iterations. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 20 of 27

instructors, both of whom are engineers. This new information changes the rules44 and means that the problem solving activity returns to the start of the Task 2 and Task 3 boxes in Figure 2. Iteration from the end of a sequence of tasks back to the start is generally added in drawings such as Figure 2 by adding a feedback arrow from the output to the input. The drawing is a representation of the iteration. However, it does not take the time factor into account. If Figure 2 is redrawn in Gantt chart45 form as shown in Figure 6 where the six tasks have been relabelled as Tasks 1 to 6 for simplicity46, the effect on schedule and cost of the iteration may be readily seen when the second set of tasks are added into the Gantt chart as shown in Figure 747. The original time from start to finish was T1 to T2; the changed time is T1 to T3 and the additional time is T2 to T3. In this instance, this Gantt chart view indicates that iteration takes place from the end of a sequence of tasks back to their start which is not always the case. Iteration may start anywhere in the sequence of tasks if deemed appropriate and may iterate back to any previous task. Knowing how and when to iterate or repeat tasks48 is an important competency in architecting the systems engineering problem solving process for a project (Kasser, 2005). Yet another iteration? The introduction to the paper mentioned that there was an unstated and implied constraint or assumption that the solution system was limited to the classroom environment and online distance mode options were out of scope. If the customer changes her mind and distance mode options become allowable or even desirable, the process must iterate another time, to examine conceptual distance mode alternatives and compare them with the already developed solutions, with corresponding cost and schedule escalations.

Afterword This section provides some comments on the following two issues.  The choice of process.  Challenging assumptions The choice of process. In this case, the customer stated the problem as the need to provide postgraduate students studying systems engineering in a classroom49. It might

44 When doing research (for a dissertation or grant) the literature review has to be ongoing (a background task) since new findings may change or even invalidate the research in process. 45 Flow charts and Gantt charts are two different views of underlying project information and should be used appropriately depending on what information is being considered, analysed or communicated. 46 This need to keep the information in drawings simple is one of the reasons for providing Work Breakdown Structure (WBS) task elements and requirements with respective identification numbers. 47 In common parlance, iteration need not be for the whole cycle to be repeated; instead, only some parts may be repeated. So Figure 7 may be overstating the case! 48 As well as when to stop work on a task; the sixth role of a systems engineer (Jenkins, 1969 page 164) 49 The problem explored so far in this paper was the customer provided problem. However, it may not have been the real problem, for a number of reasons including:  The point of education and training in systems engineering is not to please the students, although if they are enjoying the process, they will learn more effectively. Nor is it to get them to pass exams. As mentioned in a footnote, surely, the value of a systems engineering can be judged only by outcome - by the quality of the students, perhaps 3 to 4 years down the road, when they have jobs in the business and Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 21 of 27

have been advantageous if, instead of the standard systems engineering problem-solving paradigm shown in Figure 2 an alternative shown in Figure 8 had been chosen (Hitchins, 2007) page 174). This alternative process addresses the problem in more detail, which seems very important in this case and should have driven down to the root of the issue: which has teased and frustrated educators for decades! Having said that, the systems engineering problem-solving paradigm provided a rational basis around which to conceptualise the solutions to the problem as stated by the customer50. Challenging assumptions. The eighth of (Jenkins, 1969) page 164) twelve roles of a systems engineer was “He challenges the assumptions on which the optimization is based.” None of the systems engineers on this project did so, specifically with regard to challenging the solution stated by the customer. However, sometimes challenging the solution stated by the customer can lead to undesired emergence. For example, in the mid 1980’s a data processing facility at the National Aeronautical and Atmospheric Administration’s (NASA) Goddard Space Flight Center was in the middle of a facility upgrade project. The facility processed data downlinked from low earth orbiting spacecraft and was running out of capacity. Working on the project were NASA, contractor and subcontractor personnel. At an early meeting the NASA project manager drew an optical Fiber Figure 8 Combined general and Distributed Data Interface (FDDI) ring systems engineering problem- architecture on the white board and issued solving paradigms (Hitchins, 2007)

they can look back and reasonably determine what the course gave them that has proved useful. So, outcome is more valuable than output.  The content aspect was not addressed in this case other than mentioning the topics taught by each of the instructors in Table 1. There was an implicit assumption that the content was correct. However, (Kasser et al., 2008) stated that research showed that courses in systems engineering were generally not teaching the things systems engineers needed to know, they were teaching the things the faculty could teach! Now, with systems engineering students, surely we are trying to promote understanding, as much as instilling knowledge. An understanding of systems, systems thinking and systems engineering is founded on systems science and systems theory: without these, there would be no anchor. So, it seems that any systems engineering course that is going to be successful in promoting understanding of systems engineering is going to have an up-front element of systems thinking, systems science (wholes, open systems, systems approach, emergence, etc.) such that the student can see the point of it all. (And case studies of a few disasters might not go amiss). 50 The choice of appropriate process and the tailoring of the process to fit the situation is an important aspect of the role of the systems engineer. The process and the degree of formality in the documents shall be appropriate to the project. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 22 of 27

a fiat that he had just drawn the system architecture. “FDDI has a 100 Megabit data rate, it’s a neat technology to use, so we are going to use it” he said as he completed the sketch and sat down. The members of the contractor’s organisation said nothing. The subcontractor’s lead systems engineer politely suggested that perhaps they needed to do an analysis of the data rates51. The NASA project manager firmly restated that the decision had been made and the matter was closed52. Two weeks later the subcontractor’s lead systems engineer was removed from the project at the request of the NASA project manager. The stated reason was that the subcontractor’s lead systems engineer was ‘too arrogant’. In this situation the assumptions were questioned without conscious thought of the decision to perform the questioning. However, conscious recognition of the need to question the assumptions can sometimes lead to an ethical dilemma as described in (Kasser, 1995b) and Chapter 19 in (Kasser, 1995a) pages 237 to 251). Things went well during the facility upgrade design phase and at no time during this process did any of the contractor or subcontractor personnel disagree with the NASA project manager53. Several months later, when data flow problems became apparent, another non-systems engineering rethink by the NASA personnel determined that the system would use two FDDI rings in series; one for input data and one for output data54. This anecdote has been an example of how, in some situations, the customer inserts a fundamental flaw into the solution system at the start of the process and systems engineering gets the blame when the solution system does not meet the need, or the implementation suffers from technical problems and cost and schedule escalations resulting from that flawed solution.

Summary This paper has provided a teaching case study to illustrate the conceptual early stages in systems engineering using the example of tackling the problem of providing postgraduate students with an optimal learning environment as an example of factors to be considered in the conceptual early stages of systems engineering, how the seeds of doom can be planted into a project in the early stage and how recursion and iteration are invoked in the systems engineering process. Questions and comments were provided in the text, in footnotes and in the afterword to facilitate discussion in class.

51 He felt that the concept of using fiber optic connections was valid, but there were viable alternatives to a ring structure (such as a matrix or cross point switch approach), especially as they would be dealing with:  Receiving and processing real--time data.  A number of spacecraft providing the data.  Incoming data rates of up to 10 Megabits.  Several input data processing elements (workstations).  Several output data processing elements (workstations).  Output data rates of between 5 and 10 Megabits.  Data transfers between the input and output processing elements. 52 He had read about it in a magazine and so was the subject matter expert! 53 Either they didn't want to upset him and get transferred, or they knew that NASA would later pay them to fix the problems – it was a Cost Plus contract. 54 Notice they were still locked into a ring architecture solution. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 23 of 27 Conclusion The systems engineering in this case was sound in that the process was done by the book. However, the seeds of doom have been planted55 since the customer initially stated the problem as the need to provide postgraduate students studying systems engineering in a classroom with the necessary knowledge and skills components in an optimal manner. Accepting that statement, the systems engineering addressed the pedagogy limited to the classroom environment. The systems engineering also ignored the knowledge and skills components due to the implied assumption that the knowledge and skills component in the class were adequate. As a consequence, the project has a high probability of failure56. That is, the subsequent processes for engineering the solution system will deliver the wrong solution system irrespective of which systems engineering Standard the process follows and the development organisation’s level of Capability Maturity! There are lessons to be learned from this case that apply to a broad range of projects in both civilian and Defence domains. Lessons learned57 include:  Identification of the correct problem is critical. If the wrong problem is identified, the wrong solution system will be produced.  Implicit and unstated assumptions potentially comprise seeds of doom.  Solution systems can be adaptations of exiting systems or new and innovative systems.  Good systems engineering in the early stages of a project is vital.  The choice of process to tackle the problem and implement the solution is as important as the choice of solution system.  Following a perfect process can still produce a poor product. This is commonly known as garbage-in-garbage-out (GIGO).  Analysis should stop once the solution is deemed to be infeasible.  Experience, excellence and knowledge in both systems engineering and the domain are needed in a project.  Iteration needs to be built into a project schedule, the higher the level of uncertainty, the greater will be the number of iterations required. Building iteration into the schedule reduces cost and schedule overruns due to unplanned iteration.  Use appropriate views of project data to minimise misunderstandings. Gantt charts for schedules (temporal views), functional flow charts for functional views, etc. a single view does not fit all purposes.  Challenging assumptions can be an ethical challenge in itself.  Using system thinking to view the problem as a whole from different perspectives is necessary through the whole problem solving process.  Students shall not be the sole evaluators of good teaching. This can be generalised to state that users of a system shall not be the sole evaluators of conceptual replacement

55 And pointed out in the text or in footnotes. 56 Would this be a valid reason to start the Independent Verification and Validation (IV&V) activities at the same time as the solution providing activities? 57 In the classroom, students could be asked to make a short presentation on each of these lessons learned and how the lesson applies in this instance and in other situations. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 24 of 27

systems58.

Biographies ZHAO Yang Yang is currently completing a Master’s in System Design and Management at National University of Singapore. After her receiving first Master’s degree in Engineering Management from City University of Hong Kong, she worked as a Research Assistant at the Department of Manufacturing Engineering and Engineering Management, City University of Hong Kong before moving to Singapore. She is interested in system engineering, management of technological innovation and entrepreneurship strategy, etc. Joseph KASSER has been a practicing systems engineer for nearly 40 years and an academic for about 10 years. He is an INCOSE Fellow, the author of “A Framework for Understanding Systems Engineering” and "Applying Total Quality Management to Systems Engineering" and many INCOSE symposia papers. He is a recipient of NASA’s Manned Space Flight Awareness Award (Silver Snoopy) for quality and technical excellence for performing and directing systems engineering and other awards. He holds a Doctor of Science in Engineering Management from The George Washington University, is a Certified Manager and holds a Certified Membership of the Association for Learning Technology. He has also served as the initial president of INCOSE Australia and Region VI Representative to the INCOSE Member Board. He gave up his positions as a Deputy Director and DSTO Associate Research Professor at the Systems Engineering and Evaluation Centre at the University of South Australia in early 2007 to move to the UK to develop the world’s first immersion course in systems engineering as a Leverhulme Visiting Professor at Cranfield University. He is currently a principal at the Right Requirement Ltd. in the UK and a Visiting Associate Professor at the National University of Singapore. Derek HITCHINS retired from full time academic work in 1994 on medical grounds, and is now a part-time consultant, teacher, visiting professor and international lecturer. Formerly, he held the British Aerospace Chairs in Systems Science and in Command and Control, Cranfield University at RMCS Shrivenham. Prior to that, he held the Chair in Engineering Management at City University, London. Derek started as a Cranwell apprentice and retired as a wing commander from the Royal Air Force after 22 years, to join industry. His first industry appointments were as the System Design Manager of the Tornado F3 Avionics, Technical Co-ordinator for UKAIR CCIS, and UK Technical Director for the NATO Air Command and Control System (ACCS) project in Brussels. He subsequently held posts in two leading systems engineering companies as Marketing Director, Business Development Director and Technical Director before becoming an academic in 1988. His current research is into system thinking, system requirements, social psychology & anthropology, Egyptology, command & control, system design and world-class systems engineering. He has published three systems engineering books: "Putting Systems to Work", John Wiley & Sons, in 1992; "Advanced Systems Thinking, Engineering and Management," Artech House, 2003; and, “Systems Engineering: A 21st

58 Because their focus is too narrow. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 25 of 27

Century Systems Methodology,” John Wiley & Sons in 2007/2008. He inaugurated the IEE’s PG M5 — Systems Engineering. He also started the UK Chapter of INCOSE and was its inaugural president. He is an INCOSE Fellow, an INCOSE "Pioneer" and a Charter Member of the Omega Alpha Association.

References Asbjornsen, O. A. and Hamann, R. J., "Toward a unified systems engineering education", Systems, Man and Cybernetics, Part C, IEEE Transactions on, Vol. 30 (2000), no. 2, 175 - 182. Benjamin, R., "The Pentagon Wars," Home Box Office (HBO), USA, 1998. Biggs, J., Teaching for Quality Learning in University, Society for Research into Higher Education and Open University Press, 1999. Brown, D. E. and Scherer, W. T., "A comparison of systems engineering programs in the United States", Systems, Man and Cybernetics, Part C, IEEE Transactions on, Vol. 30 (2000), no. 2, 204 - 212. Carless, D. R., "Factors in the implementation of task-based teaching in primary schools", System, Vol. 31 (2003), no. 4, 485-500. Dale, E., Audio-visual methods in teaching, Dryden Press, New York, 1954. Davis, J. F., Murrell, P. H. and Davis, T. M., "On Matching Teaching Approach with Student Learning Style: Are We Asking the Right Question?," U.S. Department of Education Office of Educational Research and Improvement (OERI), Washington DC, 1988, p. 20. Drucker, P. F., Management: Tasks, Responsibilities, Practices, Harper & Roe, New York, 1973. Drucker, P. F., Innovation and Entrepreneurship, Harpercollins, 1985. Dunn, R. S. and Dunn, K. J., "Learning Styles/Teaching Styles: Should They ... Can They ... Be Matched?" Educational Leadership, Vol. 36 (1979), no. 4, 238-244. Felder, R. M. and Silverman, L. K., "Learning and Teaching Styles In Engineering Education", Engineering Education, Vol. 78 (1988), no. 7, 674-681. Felder, R. M. and Soloman, B. A., Learning styles and strategies, 2008, http://www4.ncsu.edu/unity/lockers/users/f/felder/public/ILSdir/styles.htm, last accessed 13 May 2009 Fitzgerald, R. J., Smart Teaching: Using Brain Research and Data to Continuously Improve Learning, American Society for Quality, 2005. Fleming, N. D. and Mills, C., "Not another inventory, rather a catalyst for reflection", To improve the academy, Vol. (1992), no. 11, 137-149. Glaser, B. G., Emergence vs. Forcing: Basics of Grounded Theory Analysis, Sociology Press, Mill Valley, CA, 1992. Grasha, A., Teaching with Style, Alliance Publishers, Pittsburgh, 1996. Hall, A. D., A Methodology for Systems Engineering, D. Van Nostrand Company Inc., Princeton, NJ, 1962. Hitchins, D. K., Systems Engineering. A 21st Century Systems Methodology, John Wiley & Sons Ltd., Chichester, England, 2007. Hitchins, D. K., What is Systems Engineering?, 2009, http://gallery.me.com/profhitchins/100315, last accessed 13 May 2009 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 26 of 27

Jain, R. and Verma, D., "Proposing a Framework for a Reference Curriculum for a Graduate Program in Systems Engineering," The International Council on Systems Engineering, 2007, p. 29. Jenkins, G. M., "The Systems Approach," in Systems Behaviour, J. Beishon and G. Peters (Editors), Harper and Row, London, 1969, p. 82. Kasser, J. E., Applying Total Quality Management to Systems Engineering, Artech House, Boston, 1995a. Kasser, J. E., "Ethics in Systems Engineering", proceedings of NCOSE 5th International Symposium, St. Louis, MO, 1995b. Kasser, J. E., "The Cataract Methodology for Systems and Software Acquisition", proceedings of SETE 2002, Sydney Australia, 2002. Kasser, J. E., "Introducing the Role of Process Architecting", proceedings of The 15th International Symposium of the International Council on Systems Engineering (INCOSE), Rochester, New York, 2005. Kasser, J. E., A Framework for Understanding Systems Engineering, Booksurge Ltd, 2007a. Kasser, J. E., Tiger Pro: A Tool to Ingest and Elucidate Requirements, 2007b, http://www.therightrequirement.com/TigerPro/TigerPro.html, last accessed Kasser, J. E., "Active Brainstorming: - A systemic and systematic approach for idea generation", proceedings of the 19th International Symposium of the International Council on Systems Engineering, Singapore, 2009a. Kasser, J. E., "The forthcoming Seldon crisis in systems engineering", proceedings of presentation to the INCOSE Singapore Chapter on 6 April, Singapore, 2009b. Kasser, J. E., Cook, S. C., Larden, D. R., Daley, M. and Sullivan, P., "Crafting a Postgraduate Degree for Industry and Government", proceedings of International Engineering Management Conference, Singapore, 2004. Kasser, J. E., Hitchins, D. and Huynh, T. V., "Reengineering Systems Engineering", proceedings of the 3rd Annual Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009. Kasser, J. E., John, P., Tipping, K. and Yeoh, L. W., "Systems engineering a 21st century introductory course on systems engineering: the Seraswati Project", proceedings of the 2nd Asia Pacific Conference on Systems Engineering, Yokohama, Japan, 2008. Kirton, M. J., Adaptors and Innovators: Styles of Creativity and Problem Solving, Routledge, London, 1994. Kolb, D., Experiential Learning: Experience as the Source of Learning and Development, Prentice-Hall, New York, 1984. Kuhn, T. S., The Structure of Scientific Revolutions, The University of Chicago Press, 1970. Laney, M. O., The Introvert Advantage: How to Thrive in an Extrovert World, Workman Publishing, New York, 2002. Lowery, L. L., Use of Teams in Classes, 2002, http://Lowery.tamu.edu/Teaming/Morgan1/sld023.htm, last accessed 6 February 2007 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 27 of 27

McClure, G. F., Book Review: The Introvert Advantage: How to Thrive in an Extrovert World, 2004, http://www.todaysengineer.org/2004/Nov/review.asp, last accessed 12 May 2009 Microsoft, Transforming Education: Assessing and Teaching 21st Century Skills,, 2008, http://download.microsoft.com/download/6/E/9/6E9A7CA7-0DC4-4823-993E- A54D18C19F2E/Transformative%20Assessment.pdf, last accessed 20 March 2009 Mills, H. R., Techniques of technical training, Cleaver-Hume Press, London, 1953. Myers, I. B. and Myers, P. B., Gifts Differing: Understanding Personality Type., Davies- Black Publishing, Mountain View, CA, 1980. Sage, A. P., "Systems engineering education", Systems, Man and Cybernetics, Part C, IEEE Transactions on, Vol. 30 (2000), no. 2, 164 - 174. Thissen, W. A. H., "Complexity in systems engineering: issues for curriculum design", proceedings of Systems, Man, and Cybernetics, 1997. 'Computational Cybernetics and Simulation'., 1997 IEEE International Conference on, 1997. Valerdi, R., Jain, R., Ferris, T. and Kasser, J. E., "An Exploration of Matching Teaching to the Learning Styles of Systems Engineering Graduate Students", proceedings of the 19th International Symposium of the International Council on Systems Engineering, Singapore, 2009. van Peppen, A. and van der Ploeg, M. R., "Practising what we teach: quality management of systems-engineering education", Systems, Man and Cybernetics, Part C, IEEE Transactions on, Vol. 30 (2000), no. 2, 189 - 196. Woolfolk, A. E., "Chapter 7 Cognitive views of learning," in Educational Psychology, Allyn and Bacon, Boston, 1998, pp. 244-283.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 16

A Systematic Approach Of Quality Verification For Efficient Replacement In Case Of Component EOL Takayuki Tomaru, Hidekazu Nishimura, Masaru Nakano Graduate School of System Design and Management, Keio University 4-1-1 Hiyoshi, Kohoku-ku, Yokohama-city, Kanagawa Japan [email protected], [email protected], [email protected]

Kosuke Ishii Department of Mechanical Engineering, Stanford University Stanford, CA 94305-4022, USA [email protected]

Copyright © 2009 by Takayuki Tomaru, Hidekazu Nishimura, Masaru Nakano, Kosuke Ishii. Published and used by APCOSE with permission. Abstract. Global component purchase is now common because many commercial product companies transfer their factory from domain to overseas. Many companies that manufacture copiers, printers, or car navigation systems including semiconductors and/or hard disk drives, however, need much time to verify component quality in order to replace components that have reached their End of Life (EOL) As EOL span of electronics components is short due to the rapid introduction of new technologies, many product companies have problems of excess stocks of component to keep manufacturing products and quality trouble caused by components replacement due to insufficient quality verification. This paper suggests a systematic approach of quality verification for replacement in case of components EOL utilizing Fault Tree Analysis (FTA) and Failure Mode and Effects Analysis (FMEA). This systematic approach helps companies achieve the following: 1) Component verification time becomes shorter than in past and verification cost is saved. 2) Component quality continuously improves and component failures are reduced. 3) Product system failures and component failures are reduced by feedback of compatibility failure verification. Introduction Along with moving a manufacturing factory from the domestic country to a developing country, each consumer product company must initiate global component procurement. By increasing complexity in the supply chain, overseas manufacturing increases complexity of quality verification. In consumer product companies, the period of quality verification they need is much longer and they are in the situation that cannot help replenishing a human resource for component quality verification.

For example copiers and printers have long life of about five years while product-life cycle of the semiconductor memory and Hard Disk Drive (HDD) is short, e.g., 1 - 1.5 year. For those reasons, the following problems occur for finding alternative components and the component quality verification in case of EOL: 1. In order to keep manufacturing in case of supplier’s component EOL, a product company must have excess components stock. 2 .System quality trouble is caused by inadequate quality verification of replaced components.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 16

Electrical Parts End of Life 1 - 1.5 year

1 In order to keep manufacturing in case of supplier’s component EOL, a product company must have excess components stock.

2 System quality trouble is caused by inadequate quality verification of replaced components.

Commercial Product Reliability Test Adapted system Test Period of production 3 - 4 Months 5 years

Figure 1. The problem in case of manufacturer’s components EOL

In order to always keep components for the manufacturing production preservation, the consumer products manufacturer deals with EOL by applying alternative components. While the production periods of the electronics components such as semiconductor and HDD are comparatively short term, i.e., 1 - 1.5 years, the consumer products manufacturer has to verify replacement components quality in short term. However since the consumer products manufacturer does not know how to make quality verification for replacement components, it is long terms (about 3-4 months) to develop quality verification methods. (Figure 1)

Next, it goes to reason that the consumer products manufacturer cannot deal promptly to EOL by using the cause effect diagram. (Figure 2) The following three are thought separately: 1st The period to EOL is very short. In order to win against competing products, component suppliers have to develop new products, apply new technology, and variety. That is the reason short-term EOL is caused.

2nd Quality verification needs approximately 3 to 4 months. Such a situation is generated because neither the contents of the quality verification, the testing conditions nor the quality verification criteria are enough. Overall quality verification process doesn't exist to easily decide the contents of this quality verification though construction of systematic process to develop quality verification methods is needed such as specification of system module and interfaces on consumer products side, specification of interfaces on components, weak point of reliability on components, data base of past quality trouble.

3rd The problem of the alternative switch plan is given. The quality verification plan for more efficient components switch in case of EOL cannot be settled on efficient mixture of consumer production manufacturing plan, consumer production developing plan and consumer product platform design towards the application of common components. In case of consumer product platform design, to apply common components it is necessary to take much time with the system verification (compatibility test of components and consumer production systems). It is necessary to engage efficiently in a human resource for quality

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 16

verification to manage EOL time and components stock to keep manufacturing because human resources are limited. In fact, human resources for evaluation are usually lacking because of the load of compatibility testing of components. (Figure 3, 4) As a factor, gap between sales forecasts and consumer market demands and change of maintenance component number to break down are changing. Consequently, the consumer products manufacturer cannot set an efficient quality plan for replacement.

This paper suggests a systematic and efficient approach of quality verification for replacement in case of components EOL utilizing Fault Tree Analysis (FTA) and Failure Mode and Effects Analysis (FMEA).

Much time to verification for quality Short term to EOL

Human Equipment Cooperative ties Management Resource Capacity With Supplier Economic Iss Change Uncertain quality

Organization control Requirement specifications Multiproduct Component Classified Safety regulation Development Contract Unclear design Environmental impact Product Quality problem

Competitor for supplier EOL Not efficient Replacement Quality verification Changing technology Reliability information hiding Architecture Uncertain quality verification Integrated Interface Technology

Variety of Component Manufacturing Product Products Alternative rate Plan EOL Product development Inventory control

Schedule Management Supplier EOL timing Cost Marketing forecast Cost benchmark Supplier EOL schedule Components period Replacement schedule problem

Figure 2. Investigation of long term quality verification

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 16

Period of Production (Commercial Product)

Supplier A Model A2 Model A3 Model A4 Model A5 Model A1

Supplier B Model B2 Model B3 Model B4 Model B5 Model B6 Model B7 Model B1

Supplier C Model C2 Model C3 Model C4 Verification Quality Initial Model C1

A A A A

B B B B B B

C C C Human Resources

Time Parts quality verification in the case of parts EOL

Human resources ＝ ×Number of commercial Products

Figure 3. Hard to verify Component Quality in case of components EOL

Completed product

System Verification Module A Module B Module C

Reliability Parts A Parts B Parts C Parts D Verification

Figure 4. Difference of System and Reliability verification

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 16

A systematic approach to quality verification

Entire quality verification system for components Entire quality verification system in case of components EOL is shown in Figure.5. In the execution of these systematic component quality verifications, the following effects are expected. 1,The quality verification system can visualize internal reliability characteristics of components which is called “BLACK BOX”. Therefore, components quality trouble can be prevented because potential trouble in components can be detected before consumer production with built-in component is marketed. As a result, component replacement costs can be reduced. 2. Applying quality verified components in new consumer product and existing product leads to save costs through higher volumes purchase.

Component EOL

Delivery Cost Risk Layer 1 Approach to replace components

Replace planning Cost Supplier Assessment

Planning for adapting replace Replace parts cost Only case of new trading component to current and developing ( New relationship) commercial products

Quality

Layer 2 Regulation Environment Safety

Rohs /Reach etc UL/CSS/DIN etc

Layer 3 Reliability verification Reliability Manufacturing process Supplier data Humidity Human error Specification Temperature Contamination DVT Voltage cycle etc Facility maintenance etc Components Design

Layer 4 System Compatibility ( System Verification) Mechanical Electrical Software Adapting for commercial product and components

Figure 5. Components verification overview in case of EOL

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 16

When the component quality is verified, it is possible to organize 4 layers. In the first layer, the approach is to replace components in case of EOL such as which supplier the consumer product side will choose, which model number of components they choose and cost benchmark analysis. It is very essential to choose component commonality on new developing consumer product and already developed consumer product. Optimization for consumer product developing schedule, manufacturing schedule of developed consumer product and component commonality can save quality verification cost and purchase cost because of quantity purchase expectation. The attention on latest road map of component developing schedule and components replacement planning that considers current component stock has also an influence to the consumer product manufacturing cost.

Next, Quality / Cost / Delivery (QCD) assessment of component supplier must be done to maintain consumer product quality in case of adopt new supplier. In particular, information exposure such as internal design data, component reliability data and failure diagnostics function data is required from supplier. Needless to say, component specification is needed. Furthermore, organization control of component supplier is required to ensure minimum quality trouble, cost competitiveness, and supplier component capability for market requirement.

In second layer, purchased components must meet the environmental and safety regulations. In particular consumer product companies have to verify environmental regulation data for Restriction of the use of certain Hazardous Substances Restriction of Hazardous Substances (RoHS) in electrical and electronic equipment and Registration, Evaluation, Authorisation and Restriction of Chemicals (REACH) established in Europe and safety regulation data for Underwriters Laboratories Inc (UL) established in America. In order to meet environmental regulations, as well as examining the environmental data obtained from supplier, consumer products manufacturers need verification of the content of hexavalent chromium lead and verification of hazardous substances with the X-ray fluorescence that will be important to verify the content of harmful substances. In safety data, in order to prevent consumer product from risk of ignition cased by open-short failures in electrical devices, consumer product companies have to do verification of open-short data available from supplier as well as testing supplier devices incorporated into the product to validate the supplier’s data and safety of consumer product. After implementing these two layers, verification of quality components is conducted.

Layer 3 is to verify the reliability of the components. Layer 4 is to verify the compatibility with consumer product systems which is the most important part in verifying the quality of components. In 3 and 4 layers, consumer product companies have different verification methods for quality inspection, which has also not been clarified. For more information, the authors describe layer 3 and layer 4 in the next section.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 16

Development method for components verification for quality

In "Components verification overview in case of EOL (Figure 5)" Layers 3 and 4 are the most important for shaping quality. They are also most difficult for consumer product manufacturers. So in this chapter, the following 4 methodology is discussed to be more specific about the quality verification described. (Figure 6) 1. Collection of past quality problem cases 2. Fault Tree Analysis (FTA) 3. Measures to prevent recurrence of troubles (components FMEA, system compatibility FMEA, component manufacturing FMEA) 4. Sets of reliability test conditions and compatibility test conditions to detect potential problems in internal component design.

Collecting components failure case in the past

And break database into category Customer Verification to complaint Adapt product Development Factory

Reliable verification Study document

Trouble in Trouble in Incompatible with Components design Components manufacturing Component and product

Fault Tree Analysis (FTA)

Component Component Compatible Design FMEA Design FMEA

Component manufacturing FMEA

Develop test condition for Reliability verification and component compatibility in the product

Figure 6. Development of test method for new test specification

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 16

In developing the verification method, the consumer product companies must collect past quality problem information which are based on developing the verification method. ¾ Quality trouble in factory: Cases study occurred in consumer product manufacture factory. ¾ Quality troubles in market: Component troubles case study that occur after delivery of the product to the market. ¾ Quality troubles in developing new consumer product: Component troubles case study that occur when components were built in system module. ¾ Quality troubles in verification for replacement: Quality trouble in system compatibility check and in components reliability. ¾ Quality troubles case study listed in the academic paper.

Next, based on data gathered by these failures, the cases of component and system failures can be clarified by investigating failure mechanisms of components and incompatibilities with system and components (FTA: Fault Tree Analysis). For example, as one of the failure modes for Hard Disk Drive (HDD) in personal computers include the following: 1. Contact with HDD magnetic recording head (slider) and glass media 2. Occurrence of missing slider 3. Bit of slider material scratch the magnetic recording head and glass media The fundamental problem is identifying the root case of failure such as shock and vibration has to be clarified. (Figure 7)

Figure 7. Fault tree analysis (Hard Disk Drive)

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 16

At the same time, make it clear on the failure mode, and start to discuss measures to prevent recurrence of problems. First, based on the FTA already explained, consumer product engineers have to discuss how to prevent quality failure which might occur. Brainstorming should be conducted to prevent quality failure collaborated with Quality management, design department, purchasing department, failure analysis department. (Figure 8) These are generally known as FMEA (Failure Mode Effect Analysis). The significant point is to clear failure mechanism of components, considering the severity of the consumer product system failure due to components malfunction (impact of user side) and root case of component failure. At the same time it is better to specify the verification method for preventing the recurrence of troubles. In case it is difficult to prepare sufficient budget and personnel to manage quality control because of various trouble predicted, to estimate the risk assessment: PRN（Priority Risk Number）is important to prioritize the prevention of trouble recurrence. (Tetsuji 1982, Ishiyama 1976)

Responsibility Supplier Consumer Product Failure Function Failure Impact of Sev No The cause of the Fau Manuf Desig Manuf Design Recommended response part mode failure erit . failure ure acturin n acturin Depart ( User ) y rate g Depart g ment Depart ment Depart ment ment Head data write Head Product 3 1 Scrape of clamp 1 ○○ ○1,comsumer product /data read element system abort galling due to designer should command Head function from scratch due to HDD exceeding iteration HDD to standby command Stack data recording error code of loading and sleep command to avoid Assembly media unloading Head exceeding iterration head ・HDD Stack Assembly load-unload. POWER ON FAIL 2 ,Consumer product ・R/W FAIL designer require supplier to ・FORMAT adapt stiffness property FAIL material for load-unload clamp. 2 Head element 1 ○○ 1, Verify 4M2S (Man/Machin contact with the /Material/Method) in contamination from manufacturing process organic matter about contamination. included 2, Verify filter in HDD can manufacturing catch 2um below facility contamination. 3 Shock in the 3 ○ HDD packaging design must transportation and meet following test criteria. manufactureing Test conditions: Free fall test from a height of 900mm (corner, 6 surface) Criteria: < 120G (HDD impact, 11ms duration)

Figure 8. Failure mode effect analysis ( Hard Disk Drive )

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 16

There are three major points to prevent major quality problems: 1st Activities in component design; 2nd Activities in manufacturing components; and, 3rd Activities in compatibility failure when components are adapted into the consumer product. If there is a risk of potential failure of their components, it is important that feedback be given to component manufacturers. If incompatibility system failure is also found in its consumer products, feedback to the consumer product designer will be important.

Following is mentioned about the specification example using HDD. 1st Regarding the quality trouble example of design components, the transfer rate of HDD is down due to vibration. This causes poor tracking of the magnetic recorder head on the media. This is called “Retrying”. In this case, the HDD designer changes the design such as storage recording density on the outside media which is affected by the HDD transfer rate. This is a general way for design change that is taken by car navigation and electronics consumer companies.

2nd Regarding the quality trouble example of HDD manufacturing, particle of organic chemical in the internal HDD causes quality trouble due to the attached head. Even though HDD is built in a clean room (class 100-200), it has the potential to come in contact with contamination including working wear which consist of organic compound. Cellulose containing organic compounds attached to the garment of manufacturing personnel could come in contact with HDD enclosure. Therefore the consumer product manufacturer has to verify the manufacturing process of component manufacturers, particularly Man / Machine / Material / Method / System / Space (4M2S) that has to be checked carefully for any changes. Components manufacturers have QC process charts to maintain manufacturing quality. If consumer product companies lack experience in manufacturing quality control, manufacturing process inspection based on this chart should be done.

Finally, regarding the quality trouble example of system compatibility, impedance mismatch of the HDD interface signal lines is mentioned. HDD transfer rate is increasing each year. Even though HDD transfer rate was 16.6MB/s (Multi word DMA) 10 years ago it is able to achieve 100MB/s (Ultra DMA Mode) high-speed data transfer now. (Okamura 2002) As the speed-up interface, HDD interface chip (MPU) that is mounted on PCBA and the HDD control interface chip such as Intel's South Gate is mismatched. As a result, waveform quality problem such as reflection and crosstalk occurred. To solve these problems, the systems design adapted interface timing chart (ATA standards) is significant. As examples of countermeasure, it is used to choose the resistance capacity and capacitor capacity inserted to improve the quality of the waveform.

This is quality verification system (FMEA) for components to prevent quality problems. To prevent all these problems, as over 100 action items have to be done, Priority Risk Number (PRN) is more efficient. In addition test conditions and acceptance criteria to verify compatibility of consumer product should be set. Regarding the HDD with car navigation system, test condition considered temperature and humidity fluctuation in the car, decision of transfer rate mode such as S-ATA and ULTRA DMA MODE, addressing block number for read/write command, selection of sequential or random access and decision of HDD standby, and sleep mode should be decided above consumer product specifications and operational situation of its car navigation system. After this series of quality checking process, adaption of alternative components can be determined.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 16

Redesign of consumer system and component

As mentioned in the previous chapter, quality verification measures such as collection of quality problem cases, analysis of failures, FMEA to prevent components design failure and components manufacturing failures, FMEA for compatibility with consumer products and original test program developed to verify whether consumer product can be adapted alternative components is useful for detection of component failure potential. When quality trouble occurs, we suggest specific measures to improve component quality such as redesign of consumer product and component. (Figure 9)

Two stakeholders to improve component quality can be considered. 1st The component manufacture company includes business transaction and design manufacturing department. 2nd The consumer product company includes purchasing, design, manufacturing and quality management department. In particular, which department makes a decision and which department can lead to improve quality? Quality management department and failure analysis department are much better to improve quality because they can correct more failure detail information from market and consumer product factory.

Component’s supplier Commercial product company Improving Design Improving Design

FMEA(Design) FTA FMEA(Manufacturing)

Component’s supplier Improving Manufacture

Figure 9.Improving quality for Commercial Products and Components

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 12 of 16

However, detailed investigation of the failure causes based on the failure analysis should be done and any problems with the component should be clear in advance. In this situation, FTA will be beneficial. If consumer product companies conduct failure analysis by themselves they are able to identify component manufacturing problems or component design problems, especially if they detect that component design failure rates are increasing. In order to prevent this situation, it is important to ensure a quality review collaborated with component manufacturers. (Hirotaka 2005, Mokudani 2002)

As above, the mentioned car navigation systems equipped with HDD are disadvantaged with respect to reliability because it is constantly exposed to changes in temperature and humidity. Particularly HDD is weak under humidity conditions because the lubricant on magnetic media in HDD is influenced by high humidity. This situation causes fly height instability of magnetic head on the magnetic media. Providing indication of specific reliability designs to component manufacturers in response to environmental changes such as humidity and temperature is significant. In the component quality review, design of current manufacturing components might not be improved. But providing feedback on the design quality of new developing components can improve component reliability.

Next, a case of changing the system design is described. As a specific example, HDD navigation systems have a problem with performance degradation due to vibration. Car navigation system manufacturers should clarify the level of vibration and frequency of vibration that causes HDD performance degradation. In order to prevent performance degradation, system designers in car navigation companies have to consider ways to reduce vibration level and to cut the high frequency. The point is to measure the HDD transfer rate using the consumer product system adapted damper like elastomer and to verify the effectiveness of damper.

Even though design change for mechanical structure may improve HDD performance, it is important to ensure quality reviews for system design in their company. Consumer product designers should discuss why they can not detect a problem before market, how to prevent the same problem, and request component design changes for component manufactures.

FMEA to prevent quality trouble is required by clarifying failure mechanism as well as researching past trouble. However only adding a series of these processes such as FMEA and FTA are not sufficient countermeasures to prevent consumer product system trouble; it is necessary to present a specific design method to avoid recurrent problems in the consumer product manufacturers. (Kaneko 2005) These are standardization activities in consumer product manufacturers. These processes to improve the product system and components quality are shown in Figure 10.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 13 of 16

Component’s Supplier Commercial Product Company Design Quality Control Design Brokerage Manufacture Failure Analysis Manufacture Purchase Department Department Department Department Department

EOL Info Developing product Road Map Existing products Manufacturing Schedule

Supplier Environment Assessment Safety data Approach Only New to replace Relationship Components Feed back Failure data FTA/FMEA Feed back to New Design Components FMEA

Failure analysis Reliability Manufacturing Verification

Manufacturing Process check

Component Design/Manufacturing Quality Review

System Test Compatibility

Design Improveme

Compatibility Review

OK NG Standardization For compatibility Solution for Design Quality Trouble Replace Parts Quality Trouble

Figure 10. Quality improving process in case of trouble

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 14 of 16

Conclusions This paper suggests a systematic and efficient approach of quality verification for replacement in case of components EOL utilizing FTA and FMEA. This systematic approach helps companies achieve the following: 1) Components verification time becomes shorter than in the past and verification cost is reduced. By shorting quality verification time, consumer product companies could efficiently replace components in case of their EOL. This approach achieved components stock reduction. 2) As component quality continuously improves and component failures are reduced, it is possible to realize a stable purchase of alternate components. (Reduction of purchased components cost) 3) Product system and component failures are reduced by the feedback of compatibility failure verification.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 15 of 16

Acknowledgment This work was supported in part by Grant in Aid for the Global Center of Excellence Program for "Center for Education and Research of Symbiotic, Safe and Secure System Design from the Ministry of Education, Culture, Sport, and Technology in Japan.

References Hirotaka, Ken., Takubo, Chiaki., Takahashi, Kuniaki. 2005. Reliability Design of Electronics Packaging for High-Performance Digital Equipment. Toshiba Review Vol.60 No.5. Ishiyama, Takayuki. 1976. New reliability technique. Operations research as a management science research, Vol.21, No.8. Kaneko, Koichi., Nakashima, Kenichi., Nose, Toyokazu,. 2005. Research on systematization of quality assurance in manufacture. Memoirs of the Osaka Institute of Technology, Series A, Vol.50,No.2. Tetsuji. 1982. FMEA and FTA. Operations research as a management science research, Vol.27, No.12. Mokudani, Takefumi,. Kinbara, Tatsuo,. 2002. Product Architecture and Technological Capability Building in the Automobile Parts Industry. Center for Research on Regional Economic Systems, Hiroshima University. Okamura, Hiroshi. 2002. Architecture and application of Hard Disk Drive : Principal of storage density /playback mechanism and interface. QC Publication.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 16 of 16

Biography Takayuki Tomaru is now a PhD student of Graduate School of System Design and Management, Keio University under the supervision of Prof. Nishimura. He has experience in failure analysis of electric / mechanical devices and in improving device quality with suppliers. His research area includes methodologies for quality control of manufacturing.

Hidekazu Nishimura is now a professor of Graduate School of System Design and Management, Keio University, Japan. His research area includes mechanical systems control and motion control for dynamical systems. He is now interested in model-driven systems engineering and systems design of products. He is a member of INCOSE, IEEE, JSME and JSAE.

Masaru Nakano is a professor of Graduate School of System Design and Management, Keio University, Japan. He was a principal researcher and laboratory manager at Toyota Central R&D Labs., Inc before joining Keio. He received a B.S. and M.S. from Kyoto University and a Dr. Eng. from Nagoya Institute of Technology. Dr. Nakano has studied in the fields of robotics, manufacturing system design, enterprise integration, business process reengineering, marketing research and sustainable manufacturing. He is a member of IFIP, JSME, ORSJ, JIMA, and etc.

Kosuke Ishii is a professor of Department of Mechanical Engineering, Stanford University. His research develops methods and tools to improve the life-cycle quality of products. He applies optimization and other modeling techniques to support design and manufacturing decisions in product development. He is a member of Science Council of Japan, ASME Fellow, Visiting Professor at Keio University, and Toyota Production Engineering Advisory Professor.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 9

Model for Estimating Logistics and Operations Support Costs in Command Control and Communications Projects Choon-Bong Wong, Yeow-Koon Tay, Tiong-Lee Ng Defence Science and Technology Agency (DSTA) 1 Depot Road Singapore 109679 REPUBLIC OF SINGAPORE {wchoonbo, tyeowkoo, ntiongle}@dsta.gov.sg Copyright © 2009 by DSTA. Published and used by APCOSE with permission.

Abstract. In defence acquisition and development projects, the costs of Integrated Logistics Support (ILS) and Operations & Support (O&S) typically make up a significant part of the life cycle cost of each system. Clarity and accuracy in estimating these costs are important for long- term budgeting and decision making. The paper provides a model for planners to estimate the ILS and O&S costs of a new Command, Control and Communications (C3) project as percentages to its capital investment cost. The analysis is based on a set of cost drivers and historical cost data. Introduction The Singapore Ministry of Defence (MINDEF) makes use of the Life Cycle Management (LCM) process for systems capability build-up in the Singapore Armed Forces (SAF). The LCM is a Systems Engineering Management Process that governs the system life cycle. It consists of four major periods and a transition period: front-end planning, acquisition management, transition to operations & support, operations & support (O&S), and systems retirement. One of the objectives of the LCM is to minimise the Life Cycle Cost (LCC) of each system. The LCC of a system comprises the sum of all financial resources required for its acquisition, development, ownership, operation and disposal. The major cost components of the LCC is shown in Figure 1. The highlighted cost boxes are the subjects of this paper’s discussion.

System Life Cycle Cost

Capital Investment Cost O&S Cost Disposal Cost

Hardware & Engineering & Integrated Software Management Logistics Support

Figure 1: Major Cost Components under the Life Cycle Cost for C3 Systems

Among other functions, the Defence Science and Technology Agency (DSTA) is responsible for acquisition and management of Command, Control and Communications (C3) systems for the SAF. The DSTA project management team (PMT) has to estimate the LCC (less the disposal cost) of all new systems for budget approval during the front-end planning phase. The PMT is usually conversant in estimating the cost of the system hardware and software. This estimation can be done through market survey once the set of requirements is determined. If development efforts are needed in the project, the risk of cost variation can be extended to the contractor by specifying the deliverables tracked to milestones. The cost of DSTA engineering and management services can also be estimated by considering the scale and complexity of the project. However, estimating the Integrated Logistics Support (ILS) cost and the O&S cost in a project has always been a challenge. Common ways of budgeting include rule-of-thumb percentage figures for different types of systems, or performing initial cost breakdown analysis. The former lacks clarity in derivation, while the latter normally does not accurately predict the actual costs. With a better understanding of the fundamental factors that affects these costs as percentages to the capital investment value, it is possible to provide the budget approval authority with clearer and more accurate cost estimates. This will in turn provide better projection of the life cycle cost of the project. The paper presents a structured way of estimating the costs based on an analysis of the cost drivers and historical data.

Cost Elements ILS refers to a set of resources that ensure the effective and economical support of a system throughout its life cycle. An integral part of the operational system, it is included as part of the capital investment cost. The major elements of ILS for C3 systems are system documentation, training, initial spares supply and support & test equipment (STE). These resources are normally acquired towards the end of the project phase. • System Documentation. Documentation includes operator manuals, technical manuals, software documents and all other information that is required in operating and maintaining the system. • Training. Different types of training are tailored for different target groups. Operator training equips the end users and system administrators with knowledge on system usage and configuration management. Maintenance training allows the technicians to perform corrective and preventive maintenance. In the SAF, the train-the-trainer concept is widely adopted, where instructors from military institutes develop internal training programs based on the OEM training syllabus. For complex systems, systems engineering training may also be conducted for military officers and DSTA engineers. • Initial Spares Supply. Spare parts are required to replace production parts that are in need of repair. For each system, the set of spares that is required is determined by computer simulation using inputs such as the training profile, component characteristics (e.g., MTBF) and repair capabilities (e.g., turn-around-time). Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 9

• Support & Test Equipment. STE are items required to support the operation and maintenance of the system. It includes physical tools, testing equipment, handling equipment and calibration equipment.

O&S cost refers to the annual recurrent expenditure that is needed for the maintenance of a system through its useful lifespan. For C3 systems, it is mainly attributable to DSTA system management efforts, contractors’ maintenance services, repair material, consumables, and license and subscription costs. • DSTA System Management. Under an arrangement with MINDEF, DSTA is the system manager for most of the C3 systems in the SAF. The system managers are responsible for system engineering support, maintenance contract management, system serviceability and safety, configuration management, and maintenance policies and processes. They also manage system upgrade and disposal. • Contractors’ Maintenance Services. Depending on each system’s maintenance support concept, contractors are engaged in preventive and corrective maintenance at different levels of maintenance support. Typically, at the depot level, which involves repairs using shop replaceable units (SRU), contractors are involved. Contractor services are also often used to supplement the SAF’s manpower in performing maintenance at the operator and technician levels. • Repair Material. Repair material refers to the replenishment of spare parts that are used to replace faulty parts in a system or sub-system. In cases where the system or sub-system cannot be economically repaired, repair material includes the purchase of suitable replacements. • Consumables. Consumables refer to expendable items (e.g., batteries) that are required for the day-to-day operations of the system. • Licenses and Subscriptions. Software license fees are paid to the software vendors, while telecom subscription or usage fees are paid to telecom companies for the provision of network bandwidth.

Cost Drivers To derive the ILS and O&S costs as percentages to the capital investment cost, several fundamental operational and system requirements are considered below. These requirements or cost drivers are in turn driven by socio-political and technology trends. They are independent of one another, and serve as inputs to the calculation of the ILS and O&S cost percentages. Six cost drivers are classified under the operational profile. • Training Location. Manpower costs for supporting exercises and trainings are part of the O&S cost of the system. To supplement the military personnel, civilian technicians and engineers are often employed in local exercises. Costs for in-sourcing the civilian manpower need to be considered. • Target Availability. The system availability target requirement will affect the volume of spares and manpower support that will be required for each system. For example, increasing the availability target from 90% to 99% will greatly increase the amount of maintenance spares and manpower support requirements. • Utilisation Rate. The frequency and duration that the system is used directly contribute to the O&S cost. The cost comes from maintenance services, consumables, repair material and telecom subscription services. Utilisation periods include system tests, trainings and exercises. • Support Requirement. Any requirement for enhanced systems readiness support from the contractors and DSTA personnel during utilisation period will impact the contracted manpower costs. On-site maintenance support is more expensive than off-site (e.g., phone) support. • Fleet Size. The bigger the fleet size of a technologically homogeneous system, the bigger the economy of scale in provisioning spares support and maintenance service. For example, a big fleet of soldier radios requires less maintenance cost (relative to the total investment cost of the radios) than a customised monitoring system that supports a small and unique group of users. • Telecom Requirement. If the commercial communication network infrastructure is needed for system operations, the expected annual telecom charges have to be taken into account.

Three cost drivers are classified under the systems profile. • System Complexity. In each C3 system, the number of software applications, hardware platforms and system interfaces will affect the quantities of spares and STE, as well as the contractors’ maintenance service costs. The level of complexity is a qualitative judgement call to be made by the PMT. • Depot Level Support. Most depot level maintenance support services are outsourced to the industry. The availability of depot level repair capability in the local industry will normally reduce repair cost as the local manpower rate is generally lower than that in many foreign OEM companies. Moreover, local repair capability will also reduce the turn-around-time for equipment repairs. • Software Requirement. Some C2 systems require annual subscription to licenses for software that are embedded in the applications, or that are necessary for the system to inter- operate with other C2 systems.

Figure 2 shows the relationship of each of these cost drivers to the ILS and O&S cost elements. Each arrow indicates that the cost driver contributes to the cost element. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 9

Major ILS Cost Elements

Documentation

Operational Cost Drivers Training

Training Location - Civilian or military Spares Systems Cost Drivers technicians

System Complexity Support & Test - Number of software Equipment applications, hardware Availability Target platforms and system interfaces Utilisation Rate Major O&S Cost Elements D-Level Support

- Local or overseas Support DSTA System vendors Requirement Management

Software Licenses

Fleet Size Man-efforts for - Number of technically Preventive / similar elements Corrective Maintenance

Telecom Subscription Repair Material

Consumables

License and

Subscription

Figure 2: Relationship between the Cost Drivers and the Cost Elements

Several other factors are originally perceived to affect the percentage of ILS and O&S costs for each system. However, careful observation shows that some of these factors are attributable to the fundamental cost drivers discussed above. On the other hand, data analysis shows that correlation between the rest of these factors to the cost percentages is not strong. • Capital Budget. There is a general observation that the bigger the size of capital investment, the smaller the percentage of O&S cost. However, this can be attributed to the fact that expensive projects are normally those with large fleet sizes. • Maintenance Support Concept. In general, fixed-cost comprehensive maintenance contracts cost more than contracts where charges are based on maintenance activities. However, the choice between these contract models depends on availability requirement and system complexity. Furthermore, whether civilian technicians are required will depend on the training location. • Training Environment. It was suspected that whether systems are operated in a mobile environment or in a static environment affects the reliability and supportability of the system, and that the number of different sites or platforms also affects the maintenance support. However, empirical data suggests that their impact on ILS and O&S cost percentage is not strong. • Equipment Type. Whether the system is made up of military specification or commercial grade equipment, and whether the system is acquired of-the-shelf or involved developmental efforts, do not affect the ILS and O&S cost percentage significantly.

Prediction Methodology Projects Scoring. A five-point scale is used to quantify the score of each project in each cost driver category. The score definition for each of the cost drivers, shown in Table 1, are determined by observations on the operational and systems profiles in new and existing C3 systems. The definition represents a good score spread among these systems. For new projects, the scores will be used as inputs to the formulae to determine the ILS and O&S cost percentages. As shown in Table 1, the cost percentages for telecom subscriptions and software licenses can be determined in a straightforward manner given the expected annual system usage, and hence they are simply added to the formulae (see later).

Table 1: Score of Each Cost Driver

Cost Drivers Score: 1 2 3 4 5 Operational Profile X1: Training Location Overseas 80-20 50-50 20-80 Local X2: Target Availability 90% 95% 99% 99.5% 99.9% X3: Utilisation Rate (hrs/yr) =<100 >100 >300 >1000 >3000 X4: Support Requirement Nil Phone 4 hour 2 hour Station support on-site on-site on-site X5: Fleet Size (units) >1000 >100 >10 2-10 1 X6: Telecom Requirement Estimated percentage of capital investment cost Systems Profile Y1: System Complexity Simple Slight Moderate Complex Highly complex Y2: D-Level Support Purely 80-20 50-50 20-80 Purely local overseas Y3: Software Requirement Estimated percentage of capital investment cost

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 9

Multi-Attribute Function. In formulating each of the ILS and O&S percentage costs, a multi- attribute function is used to take in the relevant cost drivers as inputs. Since the cost drivers are independent of one another, an additive function is used. An exponential factor is needed for each of the cost driver because the cost may not be linearly related to the scoring scale (as in Table 1). To prevent the function from getting too complicated for its intended users, it is defined in the following form: m Z = α X βi ∑ i i i=1 where Z is the objective function (i.e., ILS or O&S cost percentage); m is the number of relevant cost drivers; X is a cost driver; α is a cost driver coefficient; β is a cost driver exponential factor.

Model Fitting. In the above model, the coefficients and exponential factors for each of the cost drivers need to be determined. Due to the large number of unknowns and the limited set of historical data, unique optimum solutions for the coefficients and exponential factors could not be derived via simple regression analysis. Instead, the approach is first to derive the exponential factors based on management experience and the scoring scale in Table 1. For example, in determining the exponent β1 for X1 (training location), a linear scale is deemed suitable because the number of civilian technicians to be employed will increase linearly with the proportion of local training locations. On the other hand, in the determination of the exponent β2 for X2 (target availability), simulations have shown that increasing the availability from 90% to 99% will require around 5 times more spares, and hence an exponential factor of 1.5 is deemed suitable. Subsequently, the cost driver coefficients αi are estimated by fitting the model (with the derived exponential factors) with historical data in existing C3 systems. This is done by minimising the root-mean-square-error (RMSE, defined below) between the model-estimated values and the historical data values using a trial-and-error method. It is assumed that the errors are normally distributed.

2 n ⎛ ∧ ⎞ ∑⎜ Pi − Pi ⎟ RMSE = i=1 ⎝ ⎠ n

where n is the number of data points; P is the actual cost percentage (ILS or O&S); P-“hat” is the estimate of the cost percentage using the model.

Cost Estimates ILS Cost Percentage Estimate. The ILS cost of a C3 project as a percentage to the total capital investment cost is correlated to the training area (X1), target availability (X2), fleet size (X5), system complexity (Y1). The ILS cost is part of the capital investment cost. Using the model fitting technique described above, the percentage Z1 is estimated as:

1.5 0.5 Z1 = 0.2× X1+ 4.0× X 2 +1.5× X 5 + 2.6×Y1

This estimate gives a range of ILS costs from 8.3% to 17.7% for existing C3 systems. The RMSE with this formula is rather significant. This is due to possible inaccuracies in the way ILS costs were calculated in the existing systems. Therefore, this formula is recommended for use only as a rough estimate.

O&S Cost Percentage Estimate. The O&S cost of a C3 project as a percentage to the total capital investment cost is correlated to the all factors listed in Table 1 except the target availability (X2). The O&S cost is not part of the capital investment cost. Using the same model fitting technique, the O&S cost percentage Z2 is estimated as:

Z2 = 0.5× X1+ 0.1× X 32.0 + 0.1× X 42.0 + 0.5× X 51.5 + X 6 + 0.1×Y12.0 + 0.8×Y 2 + Y3

This estimate gives a range of O&S costs from 2.1% to 16.3% for existing systems. The average margin of error with actual expenditure is less than 3.5%. The O&S cost estimation is based on the year when the project is transited to O&S. The cost for subsequently years will be subjected to annual price escalations due to inflation.

Conclusion This paper provided a general analysis on how the ILS and O&S cost percentages are dependent on the identified cost drivers, and how these percentages can be estimated using the derived formulae. It provides the front-end project planner with a systematic approach to budget for the costs associated with the later phases in the project. It has to be emphasised that the model only provides an estimate of the cost percentages, since the error margin can be improved when more data points (i.e., more projects) are available in the future. The model is being validated with ongoing C3 projects in DSTA. Refining the coefficients and exponential factors in the formulae will be a continuing effort. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 9

After its successful implementation, the model can be extended to other types of systems in the SAF. However, since the cost drivers and major cost elements in the other systems may be slightly different, the model should be tuned accordingly.

Biography WONG Choon Bong is Senior Systems Manager and Principal Engineer in the Defence Science and Technology Agency (DSTA), Singapore. He holds a B. Eng. in Electrical Engineering from the National University of Singapore and a M. Eng. in Electrical and Computer Engineering from Cornell University, USA. A communications engineer by training, his experience in DSTA includes project management, software development and system integration for various communications-related projects to the SAF.

TAY Yeow Koon is a Deputy Director (Operations & Support) in DSTA. He holds a B. Sc. (Comp Sc.) degree from National University of Singapore. He has 19 years of experience in the C3 domain involving diverse users from Military (Air, Land, Naval, Joint) to Home Front Security Agencies (SPF, SCDF, ICA, PD, MOT, MOH, etc).

NG Tiong Lee is a Deputy Director (Operations and Support) in DSTA. He holds a B. Eng. (First Class) in Electrical Engineering from the University of New South Wales, and a Master of Science in Communications and Signal Processing from Imperial College, UK. His experience includes maintenance management and engineering support for Army systems in the SAF, and lead for the Business Processing Re-engineering for the Army Logistics.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 10

Systems Engineering Through The Human Lenses Lean Weng Yeoh Horng Leong Lim Defence Science & Technology Agency Defence Science & Technology Agency 71 Science Park Drive 1 Depot Road Singapore 118253 Singapore 109679 [email protected] [email protected]

Seng Guan Soh Defence Science & Technology Agency 1 Depot Road Singapore 109679 [email protected]

Copyright © 2009 by Lean Weng Yeoh et al. Published and used by APCOSE with permission.

Abstract. As countries are increasingly connected in many aspects such as trade, research and security, the actions from the individual, community and country could potentially affect other people globally. The global economy crisis and the tainted milk powder supply chain were examples that remind us of weaknesses in the complex systems could trigger massive and undesirable consequences. This paper discusses a different perspective for appreciating and practising Systems Engineering. Are systems engineers bounded by the presumed system boundary? The paper does not seek to provide an answer, but to trigger deeper thought and stimulate more questions for advancing the practice of systems engineering for the most important stakeholder, that is human.

Introduction Year 2008 was challenging and eventful. In the West, the sub-prime mortgage crisis and fall of Lehman Brothers were precursor that triggered the global financial crisis had taken millions of people by surprise. The global financial crisis raised many questions on the reliability, trust and risk of the US financial systems. On the further East, the tainted milk powder scandal in China was unfortunate that caused several infant deaths due to the poisoning. Were we overlooking important elements such that the current systems engineering practices had created boundaries that influence system thinking and mental model? How could we look at the systems boundaries, and perhaps extend the boundaries or remove them when designing systems?

Let us review the fundamentals of a system and systems engineering, so as to start off with an open mind.

System. A system can be broadly defined as an integrated set of elements that accomplish a defined objective. Many people from various engineering disciplines have different perspectives of a system. For example, some software engineers refer a system as an integrated set of computer software programs. Electrical engineer might refer a system as complex integrated circuits or an integrated set of electrical units.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 10

Systems Engineering. INCOSE defined systems engineering as an interdisciplinary approach and means to enable the realisation of successful systems. It focuses on defining customer needs and required functionality early in the development cycle, documenting requirements, then proceeding with design synthesis and system validation while considering the complete problem.

Systems Engineering also integrates many disciplines and speciality groups that form a structured process, that begin from concept to production, operation and then retirement. Systems Engineering also considers the business and technical needs of every stakeholder with the goal of providing quality products, systems and services (INCOSE). Taking a Closer Look At Systems Engineering Globalization has connected the people more closely, and the happenings at one side of the world could be felt strongly on the other side of the world. The stability of any global large-scale systems requires a closer look and a fresh perspective, so as not to repeat similar mistakes of yesterday.

One burning question would be “How are we viewing systems engineering on a global basis?”

The realization that more considerations are needed in systems engineering was not new. In the past, engineers had already been thinking about human factors, and what it means to the systems being design.

Human Factors Engineering (HFE). Application of HFE on system design is addressing interaction of the people with the tasks, machines (or computers), and environment. It takes into the considerations on the human limitations and capabilities when designing the systems, products and services. It can be applied for designing systems that require human interaction that involve hardware and software. Its objectives include ease of use, increased overall system performance and reliability, and greater user satisfaction, while reducing operational errors, operator stress, training requirements, user fatigue, and product liability. During the engineering phase, human factors engineers develop models for understanding the interaction between human and another human, or a group of humans, organisations as well as machines. (Wikipedia, Human Factors).

An example of applying HFE is designing a medical device. Kaye and Crowley highlighted three possible outcomes for a medical device. One outcome would be the device is safe and effective. The remaining outcomes for the medical device are unsafe and ineffective. These outcomes were largely determined by several factors when considering the human factor aspect. The factors were the operating environment, users characteristics and the user interface characteristics. This interaction and its possible results were depicted in Figure 1. (Kaye) Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 10

Figure 1. Interaction of HF Considerations Results in: (1) Safe and Effective Use, or (2) Unsafe or Ineffective Use

Even with HFE, human continues to be the weakest link in a system due to various inherent emotions.

In recent years, many people have continued to look beyond Systems Engineering to better solve the systems problems of today and tomorrow, and Engineering Systems is one of such field of study. The definition of Engineering System is as follows:

Engineering System. It s a field of study taking an integrative holistic view of large-scale, complex, technologically enabled systems with significant enterprise level interactions and socio-technical interfaces (Rhodes, D. and Hastings, D 2004).

The different perspectives are highlighted in Table 1. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 10

Table 1. System Engineering and Engineering System Perspectives Systems Engineering Engineering Systems Perspective Perspective Policy Viewed as fixed and Viewed as variables --can be created constraining system or adapted to optimize overall solution solution Socio- Viewed as considerations Viewed as primary in understanding technical in engineering practice the overall system Stakeholders Primary focus on customer Balanced focus on all stakeholders and end-users with impacted by engineering system -- secondary focus on other product, enterprise, environment stakeholders Focus Focus is primarily on the Focus is on product system, enterprise product system system, and environment Practitioners System architects, systems System architects, enterprise engineers, related architects, engineers, operations specialists performing analysis, project managers, policy systems engineering makers, social scientists, and many process others Vision Predictably develop Predictably develop sustainable systems with value to engineering systems with value to primary stakeholders society as a whole

By building on the work of Engineering Systems, let us explore the concept of a Border-less View of Systems Engineering.

“Boundary-less” Framework of Systems Engineering. Systems Engineer defines the system boundary to manage the scope of the system, to specify the interaction between the system and the environment, and to state the targeted environment that the system would operate in. For instance, a computational system that comprises of a set of 80 servers might be generating 80,000 British Thermal Unit (BTU) would need to be installed in a room that is equipped with cooling equipment of at least 80,000 BTU capacity. Hence, the system boundary would probably be the server room that could transfer 80,000 BTU of heat outside of the room hourly. If we would take this further, the system would release 1.92 million BTU of heat to the environment daily and this would probably not in the scope of considering the effect to other systems that are sharing the environment throughout the operations.

Figure 2 depicts a suggested framework for “boundary-less” systems engineering. We took a different perspective whereby the system, environment and human are the interacting elements. The three elements should be managed properly so that the eventual system could operate well in the targeted environment and is able to meet the human needs. Within the interacting circle, the stakeholders would have to make decisions through careful balancing act among the risk, the system value and the trust to be placed on the system. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 10

SYSTEM

RISK

VALUE TRUST

ENVIRONMENT HUMAN

Figure 2. “Boundary-less” Framework Systems Engineering

Human. Human usually form the starting point for the design and development of a system. We would shape the new systems to meet our needs and satisfy our wants.

When engineering a system, there are several alternatives that would meet the human needs. A systems engineer would usually evaluate the alternatives objectively. However, there were times when the decisions were influenced or swayed by the individual’s experiences, their belief and the value the individual will derive from the system.

In a community, their way of living, belief and value would also affect the selection of system alternative. These would translate into policies, regulations, laws, guidelines, code of practices, ethics and rules that provide the governance that every system engineer need to be aware when designing a system. Some examples of the formulated “rules” include the following: • Policies such as the International Treaty on Nuclear Agreement (e.g. Indo-US Nuclear Agreement) • Code of Conduct (e.g. Geneva Code during War Time) • Health And Safety Regulations (e.g. ECC level in radio / radar (GlobalSecurity))

Human has emotions. Their thought and action could be influenced by suspicion, happiness, anxiety, greed and fear. A user might develop fear when interacting with a new system. That user might not understand the system mechanism, its outputs, and possible safety issues. Unknowing of system reliability issues if any and the possible negative effects on the environment might affect the user’s trust level on the system. For instance, if a system might introduce fire hazard when the wrong parameters were set, the user would probably not want to risk their house to catch fire when there were absent of safety features to prevent setting the parameters incorrectly. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 10

When stakeholders were asked, “What is your expectation from the system?”, a common reply would be that they wanted a faster and better system that come with the lowest price tag. Some people might argue that the reply was driven from reasons such as the stakeholders did not understand their requirements and they just want to be “safe” when stating their requirements. It was also possible that greed might influence the shaping of their requirements. The milk powder scandal and the sub-prime crisis were some examples. When checks and safety features were not adequately designed in the system, we would probably expect a minority to exploit the gaps that could eventually lead to system failure. An example would be Nick Leeson, the infamous trader, who had exploited the unchecked risk-taking and caused the collapse of the Barings Bank in 1995.

When there was lack of surveillance for the system weakness, we could not depend on ethics to guide the human’s action. Hence, we need rules to guide the engineer for designing the system, and to guide the user operating it. We also need rule-enforcer to ensure that rules are adhered. Through empowering the enforcement unit to develop the framework and providing the governance, the setup would encourage a coherent system design and ensure that gaps would be filled and not left for exploitation.

Other considerations when designing a system include the living habit and cultural differences, which is especially evident when comparing them between the East and West. For instance, many people from the west felt that the number ‘13’ is unlucky and there were cities in the United States where the number ‘13’ were not used for the house number.

The weakest link in the system could possibly be the human who interacts with the system. Education and adequate training would raise their competency level. The established rules and governance would ensure that gaps were properly addressed and the human is given the guidance when interacting with the system.

System. There are many factors to be considered when designing a system. Some of the factors that were commonly adopted when evaluating the alternatives are as follow: • Cost Savings • System Efficiency • System Effectiveness • Expected Business Revenue • Operational Result • Reduction in Turnaround Time • Manpower Reduction • Improved Decision Quality

Global View versus Macro View. We can look at the earth from 10,000 feet above, from a mountain-top or from your house. Each viewpoint gives us a different set of considerations.

The level of viewpoint directly influences your system design. Without appreciation of the global systems, the sub-system will not factor in the relevant issues at a higher level.

All system becomes sub-system when viewed at the next higher level. How high should we go? All sub-system also becomes a system, when viewed from a lower level. How low should we go?

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 10

Taking a clock as an example, the system requires a power source of which is a battery most of the time. Its output would be the time display.

To consider the impact of the clock on its environment would need us to take a look at the form factor of the clock, the display of the time, the color/shape/sizes of the clock.

Other issues involved power saving mechanism (so as to be environment friendly) and the speed of the human receiving the visual signals from the clock. Beyond that, accuracy is another critical factor.

The above viewpoints come from different stakeholders, with different roles, responsibilities and accountabilities.

Roles, Responsibility and Accountability. A system typically has more than one person involved in it, be it the user, the manager, the designer or the developer. Typical concerns of the various parties are as follows: • CEO - shareholders return • Business Manager - ROI • Operations Manager – KPI measurement • Project Manager – Budget/Scope/Time/Quality

Having different roles will frequently dictate different viewpoints and accountabilities. Example, a business manager will focus on Return-On-Investment while a technical consultant will focus on technical feasibility. Hence a steering committee is almost always necessary to make sure that things are still progressing as planned.

More Questions in System Design. Some questions concerning the design of a system are as follow:

First, the users might not be always clear about their needs and how to work with the systems engineer on the approach to address their concerns. As the scope would probably incomplete, the consideration would be limited to the few obvious factors. Other non-obvious factors that would need further works in order to expose them for evaluating the alternatives may take longer time.

Second, it is a challenge to develop a user-friendly interface that would satisfy every user. As different users have various preferences, there are attempts to provide better sensory input and output that aim to elevate the usage experience. One possibility is to push for the Human Computer Interfaces into direct communication with the human neurons. The dream is to activate system functions through a simple thought. (Roman 2007)

Third, how can we foresee the impact of a new system on the environment? Can we always execute the system in a simulated environment before development? The potential interactions of the new system and the targeted environment are often not clear.

Fourth, a look at modern industrial housing seems to show that they are not built around human. Rather, we fit the human into the building. This is different from the olden types of housing which has ample space to roam about, both within and without. One comment from a vendor coming to Singapore was that our flats looked like power units!

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 10

Fifth, most systems are Living Systems, in which various components are in a state of constant flux. This is especially true when the systems has a long chain of users, leading to different agendas within the systems. As such, mechanism needs to be put in place to make sure that other human issues are not compromised.

Sixth, how many risks are we willing to take to implement a system? Some systems’ concepts are not proven in the real world, but can provide good payoff when implemented. Are we comfortable with only a 10% success rate for these kinds of systems?

We believed that there are no perfect systems, but systems that are optimally stable within a certain set of environmental variables. A system in use is in continuous interactions with its environment, and we need to take a closer look at the kind of impact on its environments.

Environmental Aspects. System exists in relation to its surroundings, and hence all systems have impact on their environment. Some issues are: • Eco-Environment Liability (e.g. liabilities for oil pollution (Michael G. Faure and Hui Wang 2004)) • Energy Consumption (e.g. car fuel usage – hybrid / water / exhaust) • Earth Global warming (e.g. air-con system output thinning the ozone layer.) • Pollution (e.g. factories polluting the river with its waste output.)

Nowadays, there is more awareness towards how human and their ways of living impacts the earth. This is due to the powerful effect of the media and Internet. Consumers of today are generally more informed of how products are produced by a company, and can choose to vote with their dollars.

To support environmentally friendly product, some companies refused to use ‘sweatshops’ in third world countries. These companies are also concerned with the disposition of its waste materials so as not to cause pollution to the environments.

However, there are higher business costs involved in the process and different businessmen have different thoughts on that. Hence, law enforcement is always necessary in certain cases.

How do we measure the combined value of a system when it is added into its environmental? Do we mostly compromise one part at the expense of the other parts? Conclusion Perhaps we should change our perspective and look beyond the boundary of the systems, and take a different approach via considering the interaction between the human, the system and the environment. While this effort might not build a perfect system, it could possibly be worthwhile to take this approach further for the benefits of the future generations of human being.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 10

References BBC News, “Financial Crisis Plummets Stocks”, http://news.bbc.co.uk/2/hi/business/7654025.stm, October 2008 Channel NewsAsia, “China recalls milk powder and health scandal”, http://www.channelnewsasia.com/stories/afp_asiapacific/view/379940/1/.html, October 2008 Faure, M. and Wang, H., “Liability for Oil Pollution: Recent Developments”, Environment Liability, Vol 12, No. 2, pp 55-67, 2004 INCOSE, “What is Systems Engineering”, http://www.incose.org/practice/whatissystemseng.aspx Kaye, R. and Crowley, R., “Medical Device Use-Safety: Incorporating Human Factors Engineering into Risk Management. Center for Devices and Radiological Health”, FDA, July 2000 Rhodes, D. and Hastings, D., “The Case for Evolving Systems Engineering as a Field within Engineering Systems”, Engineering Systems Symposium, March 2004 Roman, et al, “Berlin Brain-Computer Interface-The HCI communication channel for discovery”, International Journal of Human-Computer Studies, Vol 65, Issue 5, pp 460- 477, May 2007 Villareal, L., “Non-Residential Training Course, Electronics Technician”, Vol 4, Chapter 4, December 1993 WikiPedia, “Human Factors”, http://en.wikipedia.org/wiki/Human_factors

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 10

Biography Lean Weng Yeoh is Director Command, Control, Communications, Computer and Intelligence Development and Defence Systems Architect of Defence Science & Technology Agency. He is also concurrently, Deputy Director of TDSI and Adjunct Professor at the National University of Singapore (NUS). He received his Bachelor and MSc degrees from NUS in 1983 and 1987 respectively. He further obtained two Masters in 1990 and a PhD degree in Electrical Engineering in 1997 from the Naval Postgraduate School. He has extensive experience working on large scale defence engineering systems. As a systems architect, he played a key role in developing the Enterprise Architecture for defence applications. He developed systems architecting methodology for masterplanning and defence transformation. He has published several papers on Enterprise Architecture, experimentation methodology and Integrated Communications Architecture. He is also Vice- President of INCOSE Singapore Chapter, INCOSE Region VI Representative to Member Board and Chairman of Industrial Group, Institution of Engineers, Singapore.

Horng Leong Lim is a Programme Manager in DSTA. He received his Bachelor of Science (Computer and Information Sciences) from National University of Singapore in 1996. He holds a Master of Science in Systems Engineering from Naval Postgraduate School. He led the development of several large-scale C2 systems and successfully fielded the systems for the Republic of Singapore Navy. He is currently applying Systems Engineering methodologies for C2 experimentation

Seng Guan Soh is a Senior Engineer in DSTA. He received his Bachelor of Science (Computer and Information Sciences) from National University of Singapore in 1999. His current works involves designing systems for maritime security.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 14

Multi-objective Optimization of Two Functions with Three Variables and its Application

Masataka Urago, Hidekazu Nishimura and Yoshiaki Ohkami Graduate School of System Design and Management Keio University, 4-1-1 Hiyoshi, Kohoku-ku, Yokohama 223-8526, Japan [email protected], [email protected], [email protected]

Copyright © 2009 by Masataka Urago, Hidekazu Nishimura and Yoshiaki Ohkami. Published and used by APCOSE with permission.

Abstract. In order to design a good product, we have to consider many criteria such as total weight, development cost, mechanical strength, operability, user friendliness of the product and so on. Therefore, the multi-objective optimization technique is extremely important mathematical tool for industrial applications. We consider a condition of a local Pareto optimum (the multi-objective optimum) for two differentiable objective functions with three independent variables without constraints. The proposed method uses the elimination of the Lagrange multipliers. Two example problems are solved by the proposed method. Introduction In this paper, two viewpoints are descried. First, a checklist of a domain and an objective function associated with optimization problems is described. Second, a condition of a local Pareto optimum (the multi-objective optimum) for two differentiable objective functions with three design parameters is described. The first issue is explained as follows. The domain of the design parameters is constructed from the constraints and is important to optimization problems. When we treat an optimization problem, we usually seek zeros of the differentiation of given objective function. However, the differentiation of the objective function of the linear programming does not have zeros. The optimum points appear on the boundary of the given domain of the design parameters. In the case that the number of the design parameters are two or three, this property that the optimal points of the linear programming appear on the boundary can be related to the property that the optimal points of the harmonic functions appear on the boundary of the domain. Therefore, the simple checklist of a domain and an objective function associated with optimization problems is interesting. When we consider the multi-objective optimization problem, a scalar formulation of many objective functions is sometimes employed. This checklist will be useful for the multi-objective optimization problems. The second issue is explained as follows. The multi-objective optimization is extremely important mathematical tool for the product design engineering. Some criteria to make a good product are expressed as numerical objective functions such as total weight, mechanical strength of the product and so on. The others are document text and human feeling. In appropriate design phases of the product, a multi-objective optimization technique for numerical objective functions is performed to narrow down the candidates of the target product and gathers the candidates on the Pareto front. The Pareto front is a set of the multi-objective optimal points. When we try to

improve a situation on the Pareto front, we have to change at least one objective function worse. A condition of a local Pareto optimum can be written with the positive Lagrange multipliers. In this paper, the elimination of Lagrange multipliers is proposed. This elimination brings us the scalar form of the two objective functions. Some points of a domain and an objective function When we consider an optimization problem, we pick up a point from a domain of design parameters and substitute it into an objective function. If the domain is the empty set, we can not substitute the point into the objective function. Hence, it is important that the domain of the design parameters is not the empty set. In addition, the following famous theorem of the calculus is suggestive subject. We can check the existence of the optimal situation from this theorem.

Theorem A continuous function on the closed interval [0,1] has the maximum and the minimum on the interval.

Therefore, we propose a simple checklist of a domain and an objective function of an optimization problem.

The simple checklist for a domain and an objective function of an optimization problem For a domain of design parameters z If the domain is the empty set, then the optimal situation does not exist. z If the domain is not the bounded closed set, the given optimization problem is not all that good. z If the domain is the bounded closed set, the domain is considered in two categories such as the inner part and the boundary of the domain. z If the boundary of the bounded closed domain has corners and cusps, we should pay attention to them. z If the domain is the bounded closed convex set, the domain is good. The others have to be treated carefully to solve the optimization problem. For a continuous objective function over the bounded closed domain with the piecewise smooth boundary z The case of one dimensional domain (the closed interval) f(x)=a+bx b ≠ 0 the maximum and the minimum occur at the boundary of the domain. b=0 the objective function has the maximum and the minimum everywhere in the domain. z The case of two and three dimensional bounded closed domains If the objective function is the harmonic function, the maximum and the minimum occur at the boundary of the domain (Kellogg 1954). The harmonic functions satisfy the following partial differential equations. ∂∂22ff For two dimension + = 0 ∂∂xy22

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 14

∂∂∂222fff For three dimension + +=0 ∂∂∂xyz222 df2 Actually, the function f(x)=a+bx also satisfies the equation = 0 . dx2 Therefore, it can be said that the maximum and the minimum of the harmonic functions in one, two and three dimensional bounded closed domains occur on the boundary. The others have to be treated carefully to solve the optimization problem. Multi-objective optimization of two objective functions with three design parameters without constraints We only consider two totally differentiable objective functions f and g with three design parameters (,x yz ,). The multi-objective optimization makes these two objective functions small at the same time. minf (,,)x y z and g (,,) x y z (1) (,,)xyz The direction differentiation is evaluated with an arbitrary vector u .

fx(,,)(,,)+++-eee u y u z u fxyz limxyz =× gradf u (2) e® 0 e gx(,,)(,,)+++-eee u y u z u gxyz limxyz =× gradg u (3) e® 0 e The definition of the multi-objective optimum (the Pareto optimum) is quite delicate when one of gradient vectors becomes zero and the other non-zero. The constraints of design parameters also make the definition of the multi-objective optimum delicate and complex. In this paper, we only consider the case that each objective function has nonzero gradient vector and do not consider the constraints of design parameters. If the two direction differentiations Eq.(2) and Eq.(3) are negative at P(x, y, z) with some vector u , then two functions are decreased by the small shift with the directionu from P(x, y, z).

0grad¹<¥×

Therefore, we call the condition that no existence of the direction u satisfying Eq.(4) and Eq.(5) the local Pareto optimum. Condition of the multi-objective optimization We consider the sets of vector u satisfying inequalities Eq.(4) and Eq.(5) and call the sets negative regions. {uu|gradf ⋅ < 0} (6) {uu|gradg ⋅ < 0} (7)

Sometimes, the set (6) is called a cone (Fukawa, Nakayama, and Tanino 1996) when the set has the symbol of “less than or equal”. The non-existence of a vector u is written as follows.

{uuuu| gradfg⋅ <∩ 0} { | grad ⋅<=∅ 0} (8) From the geometrical interpretation of the expression (8), Eq.(9) is derived (Urago, Nishimura, and Ohkami 2008, 237-238).

gradfg= −>λ grad (λ 0) (9) We also derive Eq.(9) by the constructive way. Eq.(10) with Eq.(11) is the general form of vector u satisfying Eq.(4). uq= −+>AfAgrad , 0 (10) q ⋅gradf = 0 (11) Eq.(10) is substituted into Eq.(5). Then, we consider the following three MECE (Mutually Exclusive and Collectively Exhaustive) cases.

CASE I gradfg⋅> grad 0 We choose the vectorq0= . Then, Eq.(5) becomes the following. The inequality Eq.(5) is automatically derived.

gradgAgf⋅u =− grad ⋅ grad < 0

There is the vector u satisfying Eq.(4) and Eq.(5) at the same time. CASE I does not bring us the multi-objective optimum.

CASE II gradfg⋅= grad 0 We choose the vector q =−gradg . It is pointed out that the vector q satisfies Eq.(11). Then, the vector u becomes the following and is substituted into Eq.(5).

u = −−Afgrad grad gA , > 0 gradgAfgg⋅=−u grad ⋅ grad − grad22 =− grad g < 0

The inequality Eq.(5) is automatically derived. There is the vector u satisfying Eq.(4) and Eq.(5) at the same time. CASE II does not bring us the multi-objective optimum.

CASE III gradfg⋅< grad 0 We choose the vector q as Eq.(12). gradgf⋅ grad q =−gradg + grad f (12) grad f 2

It is pointed out that this vector satisfies Eq.(11). The vector u is substituted into Eq.(5). Then, the following inequality is derived.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 14

2 1 ⎡()gradgf⋅ grad 2 ⎤ 0grad<−g grad f 2 0>⋅() gradgf grad2 − grad g22 grad f

()gradgf⋅< grad2 grad g22 grad f (14) Eq.(14) is a part of Schwartz inequality (Nagata et al. 1990). If the vector grad f is not parallel to the vector gradg , Eq.(14) automatically becomes valid. Therefore, we can choose the constant A and construct the vector u . This is not the multi-objective optimal situation. However, the Schwartz inequality has the symbol “equal” too. This equality occurs when two vectors grad f and gradg are parallel.

III-1) gradfg=>λ grad (λ 0) We can evaluate the inner product of two vectors grad f and gradg . gradgf⋅ grad=>λ grad g2 0 This is not CASE III . CASE III-1) is automatically ignored.

III-2) gradfg=−λ grad (λ > 0) We can also evaluate the inner product of two vectors grad f and gradg . gradgf⋅ grad=−λ grad g2 < 0 This is CASE III. We use the general form of vector u written by Eq.(10) with Eq.(11). The vector u is substituted into Eq.(5) and brings us the following result.

gradgAgf⋅=−uq grad ⋅ grad +⋅ grad g 1 =−Aggrad ⋅ grad f −q ⋅ grad f λ =−Agf()grad ⋅ grad > 0 Therefore, the general form of vector u can not establish two inequalities Eq.(4) and Eq.(5) at the same time. The CASE III-2) is the multi-objective optimal condition of this paper.

Scalar expression by elimination of λ Sometimes, the Lagrange multiplier λ is difficult to manipulate arithmetically. Therefore, the Lagrange multiplier λ is eliminated from Eq.(9). ∂∂∂∂∂∂∂∂∂(,)fg f g f g g g⎛⎞ g g =−=−−−=λλ⎜⎟0 (15) ∂∂∂∂∂∂∂∂∂(,xy ) x y y x x y⎝⎠ y x ∂∂∂∂∂∂∂∂∂(,)fg f g f g g g⎛⎞ g g =−=−−−=λλ⎜⎟0 (16) ∂∂∂∂∂∂∂∂∂(,)yz y z z y y z⎝⎠ z y ∂∂∂∂∂∂∂∂∂(,)fg f g f g g g⎛⎞ g g =−=−−−=λλ⎜⎟0 (17) ∂∂∂∂∂∂∂∂∂(,zx ) z x x z z x⎝⎠ x z Eq.(15), Eq,(16) and Eq.(17) sometimes appear in the coordinate transformation (Iwahori 1967). These three expressions Eq.(15), Eq.(16) and Eq.(17) are combined into the following scalar form. The scalar form Eq.(18) is obviously equivalent to the three expressions Eq.(15), Eq.(16) and Eq.(17). The scalar form Eq.(18) is useful compared to the vector expression Eq.(9).

222 ⎡⎤⎡⎤⎡⎤∂∂∂(,)fg (,) fg (,) fg ⎢⎥⎢⎥⎢⎥+ +=0 (18) ⎣⎦⎣⎦⎣⎦∂∂∂(,xy ) (,) yz (,) zx However, we do not use the positivity of the Lagrange multiplier. Therefore, some solutions of Eq.(18) are not solutions of Eq.(9). It is important to check the solution of Eq.(18) to find the solution of Eq.(9) that is the local Pareto optimum. Conversely, the multi-objective optimal condition Eq.(9) is considered from Eq.(15), Eq.(16) and Eq.(17). From Eq.(5), one of the partial derivatives of g with respect to each design parameter is not zero. We assume the following partial derivative is not zero. ∂g ≠ 0 ∂z Then, Eq.(16) and Eq.(17) are transformed into Eq.(19) and Eq.(20) respectively. ∂∂∂∂∂(,)f gfgfg ==0 − ∂∂∂∂∂(,)y zyzzy ∂f ∂∂fg ∴ −=∂z 0 (19) ∂∂yy∂g ∂z ∂∂∂∂∂(,)f gfgfg ==0 − ∂∂∂∂∂(,zx ) z x x z ∂f ∂∂fg ∴ −=∂z 0 (20) ∂∂xx∂g ∂z

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 14

We introduce the symbol λ% defined as follows. ∂f λ% =−∂z (21) ∂g ∂z We obtain similar forms of Eq.(9) and show the forms as Eq.(22), Eq.(23) and Eq.(24). However, the positivity of λ% can not be derived from Eq.(15),Eq.(16) and Eq.(17). ∂fg∂ + λ% = 0 (22) ∂xx∂ ∂fg∂ + λ% = 0 (23) ∂∂yy ∂fg∂ + λλ%%=−∞<<∞0, (24) ∂∂zz When we derive Eq.(22) and Eq.(23), Eq.(15) is not used. Therefore, Eq.(22) and Eq.(23) is substituted into Eq.(15) in order to confirm the validity of Eq.(15) during this discussion. The validity is confirmed by Eq.(25). ∂∂∂∂∂∂∂∂∂(,)fg fg fg gg g g =−=−−−=λλ%%()0 (25) ∂∂∂∂∂∂∂∂∂(,xy ) x y y x x y y x Therefore, we can obtain the following logical relations among these expressions. gradfg=−λ grad (λλλ > 0) ⇒ grad fg =− grad ( −∞< <∞ ) ⇔ ⎧∂(,)fg = 0 ⎪ ∂(,xy ) ⎪ 222 ⎪∂∂∂∂(,)fg⎡⎤⎡⎤⎡⎤ (,) fg (,) fg (,) fg (26) ⎨ =⇔00⎢⎥⎢⎥⎢⎥ + + = ⎪ ∂∂∂∂(,)yz⎣⎦⎣⎦⎣⎦ (,) xy (,) yz (,) zx ⎪∂(,)fg ⎪ = 0 ⎩ ∂(,zx )

Example problems

Problem 1 Minimize two objective functions f and g at the same time (Nakayama, Okabe, Arakawa, and Yun 2007). 222 ⎧⎪ fx=++ yz ⎨ 222 ⎩⎪gx=−()()()555 +− y +− z

Solution ∂∂∂fff = 2,x == 2,yz 2 ∂∂∂xyz

∂∂∂ggg =−2(xyz 5), =− 2( 5), =− 2( 5) ∂∂∂xyz ∂∂fg ∂∂ fg −=⋅−−⋅−=−−+=−=2xy 2( 5) 2 yx 2( 5) 4( xyxxyy 5 5 ) 20( yx ) 0 ∂∂xy ∂∂ yx ∂∂fg ∂∂ fg −=⋅−−⋅−=−−+=−=2yz 2( 5) 2 zy 2( 5) 4( yzyyzz 5 5 ) 20( zy ) 0 ∂∂yz ∂∂ zy ∂∂fg ∂∂ fg −=⋅−−⋅−=−−+=−=2zx 2( 5) 2 xz 2( 5) 4( xzzxzx 5 5 ) 20( xz ) 0 ∂∂zx ∂∂ xz ∴ xyz==

We have to check Eq.(9). The following calculation is performed. 22(5)(0)x = −⋅λ x −λ > x λ =>0 5 − x ∴05<

75 A sample point

50

g

25

2 ⎛⎞ f g =−35⎜⎟ ⎝⎠3

0 0 25 50 75 f

Figure 1. The Pareto front of the problem 1 and the distribution of the two objective functions

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 14

To see the solution of the multi-objective optimization and the distribution of two objective functions from the global viewpoint, Figure 1 is shown. We can make the values of two objective functions on top of the Pareto front Eq.(27) small at the same time as shown in Figure 1. Once the values of the objective functions reach the Pareto front Eq.(27), we have to perform the trade off. If f becomes small, then g becomes large on Eq.(27).

Problem 2 Minimize two objective functions f and g at the same time. In order to make this example, Okabe method (Nakayama, Okabe, Arakawa, and Yun 2007) is used. 22 ⎧ 2 ⎛⎞yz+ ⎪ fxyz(, ,)=+() x 1⎜⎟ 1 − ⎨ ⎝⎠2 ⎪ 222 ⎩⎪ gxyz(, ,)=+() x 1 y + z Subject to −11≤≤x 1 ≤ yz22+≤1 16

Solution ∂+fyz⎛⎞22 =−21x⎜⎟ ∂x ⎝⎠2 ∂f =+−()xy2 1 () ∂y ∂f =+−()xz2 1 () ∂z

∂g =+2xy22 z ∂x ∂gy =+⋅()x2 1 ∂y yz22+ ∂gz =+⋅()x2 1 ∂z yz22+

∂∂∂∂∂(,)f gfgfg 2 ++ yz22 =−=+⋅xy() x2 1 ∂∂∂∂∂(,xy ) x y y x yz22+ ∂∂∂∂∂(,)fg fg fg =−=0 ∂∂∂∂∂(,)yz y z z y ∂∂∂∂∂(,)fg fg fg 2 + y22+ z =−=−+⋅xz() x2 1 ∂∂∂∂∂(,zx ) z x x z y22+ z 222 ⎡⎤⎡⎤⎡⎤∂∂∂(,)fg (,) fg (,) fg 2222 2 2 012=++=+++⎢⎥⎢⎥⎢⎥x ()(xyz ) ⎣⎦⎣⎦⎣⎦∂∂∂(,xy ) (,) yz (,) zx ∴x = 0

We obtain the candidate of the Pareto front. We have to check Eq.(9).

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 14

∂f = 0 ∂x ∂f =−y ∂y ∂f =−z ∂z

∂g = 0 ∂x ∂gy = ∂y yz22+ ∂gz = ∂z yz22+ y ∴−+yyz22 + ⋅ =0 yz22+

1 ∴xfgyz=⇒0 grad +λλ grad = 0, =22 + ≥ > 0 4

This is the local solution of the multi-objective optimization of Problem 2. The following calculation brings us the local Pareto front.

yz22+ f =−1 2 gyz=+22 g 2 f =−1 2 ∴gf=−2(1 ) (28)

To see the solution of the multi-objective optimization and the distribution of two objective functions from the global viewpoint, Figure 2 is shown. If f becomes small, then g becomes large on Eq.(28). The characteristic property of the multi-objective optimization occurs. The curve of Eq.(28) also appears in appropriate location in Figure 2.

Conclusion In this paper, two viewpoints are descried. First, a checklist of a domain and an objective function associated with optimization problems is described. Second, a condition of a local Pareto optimum (the multi-objective optimum) for two differentiable objective functions with three design parameters is described. Constraints for the design parameters do not considered. The local Pareto optimum condition Eq.(9) is derived and transformed into the scalar expression Eq.(18). The relation between Eq.(9) and Eq.(18) is clarified. Two example problems are solved by the proposed method.

2

2(1− f )

g

1

A sample point 0 0.4 1 f 2 Figure 2. The Pareto front of the problem 2 and the distribution of the two objective functions

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 13 of 14

References de Weck, O. L. 2004. Multiobjective Optimization: History and Promise. The Third China-Japan-Korea Joint Symposium on Optimization of Structural and Mechanical Systems. Kanazawa. Japan. http://strategic.mit.edu/content/view/31/79/.

Fukawa, H., Nakayama, H., and Tanino T. 1996. Linear Algebra and Convex Analysis. CORONA PUBLISHING Co., LTD. written in Japanese. pp. 130.

Iwahori, N. 1967. Vector analysis. Syokabo. written in Japanese. pp. 122.

Kellogg, O. D. 1954. Foundations of Potential Theory. Dover. pp. 223.

Nagata, M., et al. 1990. Linear algebra. Kinokuniya Shoten. written in Japanese. pp. 115.

Nakayama, H., Okabe, T., Arakawa, M., and Yun I. 2007. Multi-objective optimization and engineering design. Gendai tosho. written in Japanese.

Sakawa, M. 1996. Optimization of nonlinear systems. Morikita shuppan. written in Japanese.

Urago, M., Nishimura, H., Ohkami, Y. 2008. A multi-objective optimization with random search. The 21st computational mechanics conference:237-238. Japan: Japan society of mechanical engineers. ———. 2008. A multi-objective optimization technique with the Monte Carlo method. APCOSE conference. Japan:INCOSE JAPAN CHAPTER.

BIOGRAPHY Dr. Masataka URAGO is an associate professor of Graduate School of System Design and Management, Keio University. He received a Master degree and a PhD in mechanical engineering science from Tokyo Institute of Technology. His research interests are systems engineering, computer modeling of engineering problems and numerical methods. He is a member of INCOSE, JSME and other academic societies.

Dr. Hidekazu NISHIMURA is now a professor of Graduate School of System Design and Management, Keio University, Japan. His research area includes mechanical systems control and motion control for dynamical systems. He is now interested in model-driven systems engineering and systems design of products. He is a member of INCOSE, IEEE, JSME and JSAE.

Dr. Yoshiaki OHKAMI is a professor and currently, Dean of Graduate School of System Design and Management, Keio University. His speciality includes strategic systems engineering with application to social and technical systems, together with dynamics and control of large space systems. He has been the president of the INCOSE Japan Chapter since its establishment in March 2008. He is a Fellow of Japan Society of Mechanical Engineers, and a member of INCOSE, IEEE, CISE and other academic societies.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 15

Case Study of the Military Aircraft R&D Programs for Systems Engineering Application

Nam Kyung Ko1), Yong Soo Kwon1), Chul Whan Kim2) , Sung Eun Lee)3 1) Department of Weapon Systems, Korea National Defense University 2) Korea Council on Systems Engineering 3) Republic of Korea Air Force 205 Susaek-dong, Eunpyong-gu, Seoul 122-875, Korea [email protected] Copyright © 2009 Nam Kyung Ko, Yong Soo Kwon, Chul Whan Kim , Sung Eun Lee . Published and used by APCOSE with permission Abstract. This work conducted a case study of the military aircraft such as F/A-22, F/A-18E/F, and T/A-50 programs for applying an effective systems engineering. F/A-22 and F/A-18E/F programs were selected because the programs were developed in the same age but the performances of the programs were extremely different. And, the T/A-50 was selected due to the reason why the program was achieved in the scope of the planned cost and schedule through applying an effective systems engineering. Each program was analyzed in the aspects of the background, program planning consisting of the acquisition strategy and industrial infrastructure, cost analysis and schedule plan, and program management consisting of the program organization and contractor management, requirements and risk management, EVMS. From these analyses, systems engineering related considerations enabling development of successful systems are presented. Introduction Understanding the effect of high-tech air weapon system became apparent with current war trends as shown from the Gulf war to the Iraqi war. It will do the same in the future warfare to be endowed with the characteristics of the effects based NCW(Network Centric Warfare). Therefore, many countries are continuously endeavoring to acquire a high-tech aircraft. Countries around the Korean peninsula are also competing for development of a 5th generation stealth fighter. Unlike such trends, current domestic Korean fighter R&D program is not even sure if it will enter the technology development phase. The fundamental problems stem from the lack of a core technology and maturity for program management. Recently, the capability to manufacture aircraft increased with development of KT-1 and T/A-50 aircraft, but core technologies such as avionics and flight control are still insufficient compared to developed countries. Therefore, OO% of the trade-off of the F-15K program is assigned to a core technology transition for development of Korean fighter. Detailed efforts are, however yet to be identified for improvement of the process for program management of the Korean fighter program. Developed countries including the US are actively employing systems engineering to develop systems meeting requirement while saving cost and schedule. The domestic T/A-50 R&D program applying systems engineering techniques met a planned cost and schedule and could develop a successful system. In this respect, this work was conducted as a case study to derive considerations for applying effective systems engineering during a military aircraft R&D. To analyze the cases, F/A-22 and F/A-18E/F fighter program which were developed in the same age were selected. The performance of them was, however extremely different. Also, T/A-50 domestically developed high-tech supersonic advanced trainer was selected. Each program is analyzed in terms of its background,

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 15

planning and management. From these analyses systems engineering related considerations enabling development of successful system including military R&D program were presented.

F/A-22 R&D Program

Background. F/A-22 was initially planned as an air superiority fighter to counter the increased capability of the ex-Soviet fighter. Interrogatory about necessity of an air superiority fighter was, however brought when anticipated threats did not become in-service by the collapse of former Soviet Union. USAF from this interrogatory changed the operational concept of the next generation fighter from an air superiority fighter to a multi-role fighter and kept the program. This change of the operational concept about threats confrontation inevitably accompanied alteration of a great many requirements over the Program.

Program plan. Program plan analysis was conducted in areas of the acquisition strategy and industrial infrastructure, technology and system development strategy, cost analysis and program schedule. Details were as following. Acquisition Strategies and Industrial Infrastructure. USAF established an acquisition strategy of actively using the competition between the contractors, because USAF desired a successful completion of the program which contained numerous risks due to the change in operational concept and the industry to remain competitive even after the development of the ATF(Advanced Tactical Fighter). It resulted in a competition between Lockheed Martin(below, LM)/General Dynamics(below, GD)/Boeing team’s YF-22 and Northrop Grumman(below, NR)/McDonnell Douglas(below, MD) team’s YF-23. At 23rd April 1991, USAF selected the YF-22 of the LM-GD-Boeing team, thus putting as end to 4 years and 7 months DEM/VAL(Demonstration & Validation) phase of the program despite numerous technical difficulties. However, the existing industrial infrastructure did not mature so much in order to support the performance needed by USAF, because the F/A-22 program was the program that develops a new high-tech fighter. Accordingly, F/A-22 program did not acquire a stable supplier to delivery high-tech parts and systems. Such an unstable industrial infrastructure served as an element to raise cost and delay schedules during the EMD(Engineering & Manufacturing Development) phase. Technology and System Development Strategies. The state-of-the-art F-22A program included numerous challenges that the high performance such as maximized stealth performance, high-tech integrated avionics, and a high powered propulsion system being capable of super cruise and thrust vectoring control should be resolved. USAF established the strategy to develop such technologies and systems simultaneously despite the accompanied high risk when developing state-of-the-art new technologies. Also, USAF planned to overcome the inherent technological risk through developer competition. USAF persisted in the single step development acquisition strategy to F/A-22 program because successfully achieved previous fighter programs and judged the high-tech technology, development and integration of the system, and the risk of synthesis course optimistically[1]. Cost Analysis and Schedule Plan. Initial estimation of feasible costs and schedule are important for the program plan to be aligned in the right way. However, cost analysis and schedule plan were estimated so low in initial because the F/A-22 program management and development team neglected the risk associated with system development and integration in terms of a lot of requirements alteration by changing the operational concept of F/A-22. Accordingly,

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 15

CBO(Congressional Budget Officer) and GAO(General Account Officer) showed doubts on the cost analysis and feasibility of the schedule in the program early. These unrealistic cost analysis, schedule plan and high risk in the F/A-22 program resulted in decreased numbers of production and several in-depth audits.

Program Management. Program management analysis was conducted in areas of program organization and contract management, requirements and risk management, and Earned Value management. Details were as following. Program Organization and Contractor Management. F/A-22 program was the first of the programs to apply the IPTs. However, the period of the F/A-22 program’s DEM/VAL phase was extended 2 years because a previous national defense acquisition organization was converted into IPTs and employed a new procedure. Also, the F/A-22 development team consisted of LM, GD and Boeing. They participated in F/A-22 program by partnership. LM was experienced in stealth aircraft development such as the F-117 and GD experienced in fighter production such as the F-16. And, Boeing possessed revolutionary aircraft manufacturing techniques. However, EMD phase tasks were equally shared by fighter’s main part without considering each company’s specialty or experience unlike DEM/VAL phase tasks. It caused cost growth and schedule delay of the F/A-22 program. Also, F/A-22 program resulted in difficulties in program management because F/A-22 program was developed while they kept partnership between contractors. Requirements and Risk Management Many requirements alterations were accompanied due to the change in an operational concept in The F/A-22 program. The cost growth and schedule delay happened because alterations of USAF’s requirements about the fuselage and stealth performance caused frequent design alterations in 1992. Tendency of the fuselage weight changes was shown in figure 1, why USAF altered requirements for the fuselage during system development phase[2].

Figure 1. F/A-22 Airframe weight changes over Time

Also, the program should develop not the derivation type of existing engine but the new type of engine that fulfilled requirements such as a super cruise and thrust vectoring required by USAF. And, F/A-22 program invested a lot of cost and schedule in order to meet requirements for developing the integrated avionics that can integrate the information provided by the new electronic phased array RADAR and individual avionics. The cost increased during the F/A-22 EMD phase for integrating major avionics was represented in Figure 2 [2].

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 15

Figure 2. F/A-22 Avionics Cost Growth by Component

As mentioned in the program plan, F/A-22 program was suffered many difficulties for an obscure responsibility in the course of integrating individual avionics, because the program equally assigned tasks to contractors without considering their specialty and experience in bulk. Also, LM and US government handed over program management and design supervision task to reduce program cost from the Burbank plant to the Marietta plant which produced and designed the cargo aircraft mostly. However, the Marietta plant lacked the design team to conduct a technology and production process required for a high-tech fighter and lacked integration experience of new parts or aircraft. Therefore, systems assembly and prototype delivery were more delayed than planned period. Earned Value Management. EVMS(Earned Value Management System) was employed to control the status of program progress, and to manage integrated cost and schedule[3]owever, EVMS effects were limited due to main problems such as unrealistic initial cost and schedule estimation in the program. During the EMD phase, the program was experienced in four re-phases that adjusted schedule and scope of the program. Also, two rebase-lines that modified the difference between EV and AC to “zero” took place. The status of the program progress was shown in Figure 3[2].

Figure 3. The F/A-22 Program Used Contract Performance Goals to Measure Progress

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 15

F/A-18E/F Program

Background. In 1988, USN initiated the ATA development program to substitute the A-6 attacker. However, the ATA(Advanced Tactical Aircraft) development program was cancelled due to cost growth of 1 billion dollars and schedule delay of 18 months in Jan 1991. Thus, USN decided to upgrade the existing F/A-18C/D in order to reduce cost and risk of the next generation supersonic fighter bomber discussed in the early 80s.

Program Plan. Program plan analysis was conducted in areas of acquisition strategy and industrial infrastructure, technology and system development strategy, cost analysis and program schedule. Details were as following. Acquisition Strategies and Industrial Infrastructure. Unlike competition composition of the F/A-22 Program, F/A-18E/F program contracted uniquely with MD to minimize risks from the cost excess and schedule delay based on the development experience of F/A-18A/B/C/D. MD selected NR as sub-contractor. Also, MD-NR team could stably conduct the program in the course of technology and component development, design, and manufacture by selecting sub-contractors with plenty of the previous program experience. Such an effective employment of the industrial infrastructure, the tasks allotment considering contractor’s specialty, and smooth coordination between contractors helped the challenges resolved and risks identified across the program. Technology and System Development Strategies. F/A-18 E/F program selected an Evolutional Acquisition Strategy that considered technology maturity of existing industrial infrastructure rather than single step development acquisition strategy employed in the F/A-22 program. The examples of the Evolutional Acquisition strategy were as following. Over 90% of Avionic systems of the initial production type in the program was designed identically to the F/A-18C/D version considering technology maturity of the related industrial infrastructure in order to minimize risk. F/A-18E/F program also established a back-up plan to use the avionics of F/A-18C/D in case avionics development failed. Thus, the avionics development did not affect on the entire program. Such avionics development program considering back-up plan on the technology maturity and development failure did not cause any schedule delays. Revised avionics included not only digital cockpit panels, FLIR, and electronic phased array RADAR, but also reconnaissance pod, JHMCS, and new ECM equipment added in recent. Schedule and cumulative cost for the F/A-18E/F avionics development was shown in Figure 4[2].

Figure 4. The F/A-18E/F Avionics Development Program

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 15

Cost Analysis and Schedule Plan. Main focus of the program was to minimize cost, schedule and risk. So, IPTs consisting of experts from various areas conducted tasks by coordinating with not only team members related directly to the development but also stakeholder related with the whole system lifecycle to produce the exact cost and schedule estimates. The estimates produced through systematical systems engineering were continuously adjusted during the program period. The program plan based on the efforts became the successful foundation of conducting the program as managed by the EVMS.

Program Management. Program management analysis was conducted in areas of program organization and contract management, requirements, risk management and Earned Value management. Details were as following. Program Organization and Contractor Management. F/A-18E/F program has been underway before IPTs was officially implemented to the defense acquisition program in 1995. However, the program could improve the maturity of the IPTs operation through lessons learned from failure of the F/A-22 program and the A-12 program. The F/A-18E/F development program could define requirements as including a broad range of the stakeholders. Therefore, requirement alterations of the design process latter which largely affect a cost and schedule could be reduced. The number of the blueprint revision in the E/F program was actually decreased more over 50% than that in the A/B version program[4]. Unlike F/A-22 program, F/A-18E/F program could be clear on the responsibilities for the contractor’s tasks because of keeping the evident relationship between main contractor and sub. Thus, the program could conduct more effective program management through allotment of the tasks considering contractor’s experience of the F/A-18C/D development and specialty. Requirements and Risk Management. USN actively participated in defining and validating requirements. And, the development team previously identified potential risk factors of cost, schedule, fuselage weight etc. through the experience of the F/A-18C/D development. By an example, USN could effectively resolve the unexpected problem, such as the wing drop which occurred in the course of the program, through close coordination with not only developer but also stakeholder related with the program. Thus, the F/A-18E/F program could proceed without outstanding schedule delays and cost overrun through the cooperative efforts by USN, developers and stakeholder. Also, overall risks of the program were reduced because the program required the least stealthy performance and selected a derivation style engine of the existing engine considering a current technological maturity. Earned Value Management. EVMS was used to effectively manage the program. USN conducted daily telephone and weekly video teleconferences with development team to supervise and manage a cost and schedule progress status of the developer. While the F/A-22 program received numerous audits and adjustments, the F/A-18E/F program was able to progress without any outstanding incidents because of keeping the close cooperation structure with the developer. The progress status of the F/A-18E/F development program was described as shown in Figure 5[2].

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 15

Figure 5. The F/A-18E/F Used EVM Data to Measure Progress

Some cost growth occurred in the area of development for the fuselage, engine and avionics. However, total development cost was used less than planned cost through reducing in the cost of the systems engineering and test and evaluation. The trends of the cost growth for major systems was indicated in Figure 6[2].

Figure 6. F/A-18E/F cost growth trends for major systems

T/A-50 Development Program

Background. T/A-50 program was to develop domestically the first supersonic advanced trainer and light attacker operated by ROKAF in the 21st century. Also, the program was the first to research and develop training and logistics systems that was needed for an aircraft operation and maintenance[5]. However, the program experienced numerous difficulties due to lack of common understanding for the domestic development, favor for direct purchase, and pessimistic view towards a development capability.

Program Plan. Program plan analysis was conducted in areas of acquisition strategy and industrial infrastructure, technological and system development strategy, cost analysis and program schedule. Details were as follows.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 15

Acquisition Strategies and Industrial Infrastructure. The T/A-50 program selected an acquisition strategy to acquire core technology in domestic immature areas by trade-off. The program acquired an advanced trainer design, simulator and test pilot education technique through trade-off of HAWK program. And obtain advanced trainer and light attacker technology through the trade-off of KF-16 fighter program. The T/A-50 development program selected international collaboration development strategy with international firms that have an advanced technology rather than capabilities and resources of a domestic development. The reason why international collaboration development was selected was to reduce the risk of development through the an advanced technology and experience. Not only could it reduce the unit cost by production in mass, but also improve international recognition and reliability in the export market thereafter. The development cost of a military aircraft is paid fully by the government. But 70% of the cost was planned to be paid by the government and the remaining 30%(KAI 17%, LM 13%) by the contractors in the T/A-50 program. It was to reduce the budget constraint of the government during the initial phase and motivate the contractor’s participation for a successful program. Investments of domestic developers were to be reimbursed during production phase. But LM’s investments were to be reimbursed 50% during production and the rest upon export. It led LM to participate actively in export marketing. T/A-50 program was conducted by KAI which had experience in KF-16 production and LM. The program could proceed in a stable industrial infrastructure through using standard or F-16 parts. Technology and System Development Strategies. T/A-50 program selected Evolutionary Acquisition Strategy to develop A-50 light attacker through a gradual improvement of the T-50. Also, the program pursued the strategy to purchase existing parts or to modify and develop the parts if inevitable. It was to minimize technical risks. The 52% of the components were used from existing parts, 33% modified, and 15% newly developed. Also, 74% of the existing parts were used from the F-16 components, which increased compatibility and s logistics efficiency. The program developed an aircraft, ILS and ground training system in an integrated manner through applying systems engineering. It was the origin to develop the systems through going abreast system development and production phase. Therefore, the program conducted a system development applying “An Advanced Trainer R&D Guideline” which was established apart from the domestic acquisition regulation. Cost Analysis and Schedule Plan. The cost of the T-50 program was produced with a reasonable cost scope considering an aircraft speed and estimation of the interrelationship with other aircraft development cost in the initial exploration development. The development schedule was derived from cost planning, organization operation size, task requirements etc. The estimates produced were continuously corrected and complemented during phases like reshaping, detailed equipment selection point etc. The continuous schedule and cost estimation formed an appropriate foundation of system development plans and adequate management activities, and became the key factors that could conduct the program successfully.

Program Management. Program management analysis was conducted in areas of program organization and contractor management, requirements and risk management, and earned value management. Details were as following. Program Organization and Contractor Management. T/A-50 system development program was conducted by ADD leading exploration and system development just like previous weapon system developments. But, a production was executed by development company participating as the

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 15

company of the prototype manufacture in the system development phase. Thus, LM claimed that the system development program was changed from ADD’s lead to company’s lead, because two fold program organizations increased the ambiguity of the responsibility, cost growth and schedule delay. So, the program was managed by ROKAF, and the development task was conducted by KAI as main contractor with LM sub-contractor. The program was conducted ultimately by a unique procedure such as “the international collaboration development of ROKAF’s management and contractor’s lead”[6]. ROKAF established a simple and clear management system and employed an effective IPTs to minimize schedule delays occurring due to delays in decision making process. Also, they were able to manage an effective program and to make a rapid decision thorough giving a field management organization a decision authority. Not only the program manager organization but also the development team was granted flexibility by renovating the organization which established a task force to facilitate an adequate tasks execution. Therefore, ROKAF program management organization and KAI development were changed almost once a year during the 8 years of T-50 system development phase. Considering specialty and experience of the contractors, technology support, avionic and flight control task for the overall development were conducted by LM, where fuselage design and manufacture, system integration and final assembly tasks were executed by KAI due to requiring a technology maturity. Requirements and Risk Management. ROKAF understood an importance of ORD(Operational Requirements Document) playing the most important role as the metric of test and evaluation phase. So, they sent ROKAF personnel to Portsworth during concept design phase and prepare the ORD with developers. Such ORD based on systems engineering activities was continuously modified and complemented during development phase through a tradeoff study. However, detailed conditions were not established for flight testing areas on system development contract. It caused conflicted views about the program completion between ROKAF and contractor. So, they needed additional times to adjust it, and thus result in 4 months delay in system development period. The occurred problems during the period were, however resolved through a close management way such as a periodic phone and video teleconference. Also, managers of ROKAF and KAI directly visited a field and conducted risk mitigation activity as supervising and managing execution status about present issues and action plans. In the case of a landing gear which caused a schedule delay as one example of risk mitigation activities, The program management team sent a senior engineer to France and UK’s Messier-Dowty to meet planned dates by checking on daily progress status. The program could systematically manage development information through an establishment of the aircraft development information management system. Also, it could reduce a cost and schedule needed to manufacture a real mock-up as using standardized data, analysis system and computer mockup based on 3 dimensional configuration models. Earned Value Management. The program appropriately employed various methods like Milestone Chart, PERT/CPM, Network Cost Chart to manage schedule in the aspects of the fuselage, detailed system etc. Thus, the program could manage a progress earned value in comparison with a planned schedule and cost, and detailed schedule was adjusted by monthly check. The revision of the detailed schedule was executed by activities of understanding a cause and establishing means to counter schedule delay. The areas of the main problems were resolved by daily meeting with senior developers and activities of earned value management based on earned value in comparison with planned.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 15

Considerations of the Systems Engineering for Military Aircraft Developments

Application of the Evolutionary Acquisition strategy. An evolutionary acquisition strategy applies a maturity technology to a subject system through the technology readiness assessment and technology readiness level, and thus is to rapidly supply needed systems with users on a required time. So, the difficulties of the core technology development were reduced by applying a mature technology to system development through the technology readiness assessment and technology readiness level. Thus, the strategy could have advantages to reduce cost growth and schedule in the overall program[7]. However, the strategy should rapidly conduct the development of process and tailoring due to developing systems partitioned through a technology maturity. So, the strategy could have defects that increased the quantity and complexity of the configuration management and should require an additional organization to manage the performance improvement of each program, because it was required to develop multiple configurations occurred according to developing the systems partitioned. It is very important to apply activities such as continuous requirements definition, TRL and TRA to the evolutionary acquisition strategy to develop a military fighter, when a long term is required to develop a military fighter. Also, the activities of the systems engineering are essential to successfully apply the strategy.

Early Participation of Military user in Development Program. A life cycle cost can be greatly varied according to how the requiring military and developers agree in requirements in the initial design phase. If clear requiring military’s requirements are delivered to the developers and the requirements are accurately designed by developers, the factors that change requirements can be reduced. Thus, it could increase a success rate of the development, and prevent a cost growth and schedule delays. The clear establishment of the coordination items with the developer is necessary to induce an early participation of the requiring military. Also, collaborative analysis of the requiring military and developers are essential to design a system and to decide a system specification selection to a high performance weapon system such as a military aircraft. The coordination items of the military and developer organization were described in each development phase as shown in Table 1.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 15

Table 1. Coordination requirements in each development phase Section Concept Exploration Pre-systems Acquisition System Acquisition - Establish development plans - Apply field experience - Development for critical characteristics during management Developer technology/parts development concept - Technology test - Analyze employment establishment evaluation support elements - Coordinate opinions on design - Required Operational - Participate in - Review critical parts Capability review development demand - Core technology management - Provide field experience Military demand - Participate in and user Recommendation technology characteristics - Establish ILS test - Review test evaluation dvelopment - Review measures scope specification(draft) - Review scope of specification

As indicated in Table 1, it is necessary to establish the detailed coordination items with the developers in each development phase. Also, an early participation of the requiring military is essential to successfully develop hi-tech weapon systems as a military aircraft.

Systems Engineering based on Cost Analysis, Earned Value and Program Management. Systems engineering is a common basis for communication, and enables a concurrent engineering development of products and processes. It also ensures a sound progress of the technology tasks based on planning, tracking, and mutual cooperation during the system development phase. It is described in Figure 7 how systems engineering integrates a cost analysis, earned value management, an activities and functions of the program management for an efficient and affordable program execution[8]. Details were as following.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 12 of 15

Figure 7. The Relationship of the Systems Engineering based Cost Analysis, EVM and Program Management

The systems engineering activities can be divided into pre-system acquisition and system acquisition. The core activity of the systems engineering is the initial phase activity that defines a concept in the aspects of the operation and function. When a favored system is selected through AoA(Analysis of Alternatives) based on defining an operational concept, a cost analysis for life cycle estimation is conducted based on WBS. The life cycle cost and requirements are defined by each risk analysis. When the program is entered in an official phase of the system acquisition, an acquisition plan is established based on a requirements definition and WBS becomes more detailed. Such activity of the systems engineering yields an estimation of the acquisition cost and the result of the cost estimation and integrates the activities of the program management with a performance measurement baseline of the EVM. Unlike a previous way that the programs in the existing acquisition process were entered in an official system acquisition phase after risk reduction activities, it means that a risk analysis is occurred at every phase. Program managers can successfully manage programs because a risk analysis and realistic cost analysis and schedule planning are yielded through the activities of the requirement definition, cost estimation, acquisition plan, and EVM performance measurement baseline review.

Establishment of the Hierarchic Integrated Product Teams. IPTs needs to be hierarchically formed to achieve efficient tasks in the task environment conducting the development of the integrated product and process simultaneously. IPTs can be described in the aspects of the execution IPTs and direction and review IPTs. The Hierarchic structure of the IPTs is illustrated as shown in Figure 8[1].

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 13 of 15

Figure 8. Hierarchic IPTs Structure

An execution IPTs corresponds to the lowest phase and is composed of the government IPTs that supports program managers and contractor IPTs that executes unique tasks allotted by WBS. Main activities of the IPTs are to conduct activities such as a program planning and preparation of milestone application and RFP(Request for Proposal). Direction and review IPTs is composed of integration IPTs that conducts the tasks of the action level, overall IPTs, and the supreme decision organization reviewing major decision points. Each IPTs functions are as following. The main functions of the action level IPTs are, to develop strategy required by PM(Program Manager) and support program planning phase, to establish IPTs planning activities and milestones, to propose requirements of the applied documents and milestone, to early review and provide inputs to documents, to adjust members of the overall IPT and activities of the action level IPT, to assess and solve problems through using appropriate means, and to simultaneously acquire major parts and documents to be applied. An overall IPTs is a IPTs of the department of defense level. The main roles are to provide a policy of the acquisition strategy and solve problems suggested by an action IPTs and integration IPTs. A supreme decision organization reviews the input of the overall IPTs, solves the problems and conducts role deciding whether the program should be entered to the next phase or not. Such operation of the hierarchic IPTs enables a rapid decision making and concurrent engineering development through conducting a unique role corresponding to supreme decision making organization with execution and action IPT. It results in an efficient program management as a reduction of the development period.

Establishment of the Integrated Information Environment. A systems engineering consists of numerous activities centered on IPTs across various sections, and such outputs are continuously modified and verified. Systems engineering process ensures the traceability through each phase documentation. Thus, the preparing and managing document are important to manage requirements. The interrelation of the related documents is more complex due to increasing documents to be managed in the process of the high-tech system development like a military aircraft. Thus, a standardized systems engineering process and management tool need to be developed in order to share all information and documents related to development. And, a national integrated information environment should be established to connect all related organizations. The most suitable information can be acquired by an integrated information environment through real-time information sharing. Also, an integrated information environment can support flexible

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 14 of 15 tasks execution related to weapon system acquisition by managing documents through electronic documents, providing traceability related to requirements changes, and minimizing administrative delays between activities.

Conclusion

This work was conducted as a case study to derive considerations for applying effective systems engineering during military aircraft R&D. To analyze the cases, F/A-22 and F/A-18E/F fighter program which were developed in the same age were selected. The performance of them was, however extremely different. T/A-50 domestically developed high-tech supersonic advanced trainer was selected also. Each program is analyzed in terms of its background, planning and management. From these analyses systems engineering related considerations enabling development of successful system including military R&D program were presented. Details are as follow; application of the evolutionary acquisition strategy, early participation of the military user in a development program, systems engineering based on cost analysis, earned value and program management, establishment of the hierarchic integrated product teams, and establishment of the integrated information environment.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 15 of 15

References

1. Y.S. Kwon, 「New Systems Engineering Primer」, Iwork Book, 2007. p. 57, p. 271. 2. O. Younossi, et al.「Lessons Learned from the F/A-22 and F/A-18E/F Development Programs 」, Rand Project Air Force, 2005. pp. 33~51. 3. DoD,「Earned Value Management Implement Guide」, DCMA, 2006. p. 2. 4. Y.S. Kwon, “Case Analysis and Prospect for Systems Engineering Application in the Weapon Systems Acquisition”, Security Research Series 4th Issue,「Information Age Information Technology and Defense Management Plan」, National Defense University, 2003. p. 161. 5. H.W. Lee et al, 「The real Systems Engineering Prepared to be based on T-50 Trainer Aircraft Development Experience」,Chung Moon Gak, 2007. p. 3. 6. J.H. Lee, 『T-50 was produced by this Method』, Knowledge Industry Company, 2006. p. 197. 7. AFMC, 「Evolutionary acquisition strategies」, USAF, 2005. p. 8. 8. Y.S. Kwon, “Application Plans of the Systems Engineering in the Phase of Defense Acquisition Program”, Defense Quality Management Journal Issue, Defense Agency for Technology and Quality, 2008.6, p. 33.

Biography

Ko. Nam Kyung is a doctor candidate of Korea National Defense University. Mr. Ko has studied in improvement of systems engineering and process for 7 years, including defense acquisition process and “system of systems” systems engineering.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 13

Tracking the Technical Critical Path: Systems Engineering Meets Project Management

James R. Armstrong Stevens Institute of Technology Castle Point on Hudson Hoboken, NJ 02070 Copyright © 2009 by James Armstrong. Published and used by APCOSE with permission. Abstract. The critical path in the project schedule, the path with the least slack, has been a mainstay of project management for many years. Along with earned value techniques, the project manager uses this method to track schedule and cost progress. The term Technical Critical Path appears sporadically as the sequence of activities needed to develop and incorporate critical technologies that will make the program successful. However, the connection between these concepts is not clearly defined. At the same time, much has been written about Technical Performance Measures (TPMs) but it has been confined to the technical management side of systems engineering. This paper will address how the two worlds of systems engineering and project management should be integrated to combine the technical, cost, and schedule performance. Also addressed are the correct implementation of TPMs and several problems encountered in current usage. Project Management Focus The Iron Triangle. The three attributes of cost, schedule, and technical performance have long been the basis of project management. However, the emphasis in project management has been on methods to track cost and schedule. As shown in figure 1, the WBS, schedules, and earned value are structured to address these management concerns.

EAC Budget Baseline Management Reserve

PM Baseline $$ Plan WBS Actuals SV CV xxxxxx xxxxxxxxxx Cost xxxxx xxxxxxxxxxxx xxxxx xxxxxxxxxx EV Xxxxx xxxxxxxxxxxx xxxxx xxxxxxxxx Time xxxxx xxxxxxxx xxxxx xxxxx xxxxx xxxxxxxxxx xxxxx xxxxxxxx xxxxx xxxxxxxxxxxxx xxxxx xxxxxxxx xxxxx xxxxxxxxxxx Schedulexxxxx xxxxxxxxx Event xxxxxxJANFEB xxxxxxxxMARAPRMAYJUNJULAUGSEPOCTNOVDEC ? xxxxx xxxxxxxx Milestonesxxxxx xxxxxxxxxAWD PDR CDR TRR FCA HW Design /PCA HW Fab HW Test Schedule Technical SW Design SW Test/Int HW/SW Integ System Test

Figure 1. The Iron Triangle with Management Emphasis In theory, the WBS tasks should have technical objectives. In practice, this is not often well

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 13 defined. In some cases, the specification requirements are referenced, but may not actually be what is expected at the point in the program where an early design is analyzed or prototype tested. The use of TPMs can provide a better linkage as will be described in the following paragraphs. Critical Path Basic. The path in a network schedule with the least (or most negative) schedule slack is the critical path. This concept has long been used for managing the highest risk schedule events on a project. A project may have more than one equally critical path or management may decide to track more than just the most critical path. Tracking the three most critical paths addresses the risk that a problem may appear on a path that is close to critical and overtake the original path without warning. Figure 2 is from a project where the critical path analysis identified multiple parallel critical paths leading to fixed events. After identification of this situation, action was taken to address this significant project risk. In this instance, multiple paths had to be monitored throughout the program.

My Integrated Schedule Q2 1998 Q3 1998 Q4 1998 Q1 1999 Q2 1999 Q 3 1999 Q4 1999 Q1 2000 Q2 2000 Q3 2000 Q4 2000 Q1 2001 Q2 2001 Q 3 2001 Q 4 2001 Q1 2002 Q2 2002 Q 3 2002 Acquisition Requirements Development 3 90 days 06/29/19910/30/199 0 days

Bid/Proposal/Award Prep./Review of SIR Released Cont ract A war d 4 5 days 5 0 days Ac tiv it ies 7 0 da ys 10/26/19910/30/199 10/30/19910/30/199 6 45 days 01/15/19901/15/199 0 days 0 days 11/02/19901/01/199 0 da ys 0 days

Design & Produ ction

Build 1 Host/DLAP S/W FCA/PCA Complete (HCS Host A5f 1.2 CodeHost Cut Integration/keysit off e National Delivery A5f1.2 Development SW /ProATN DL AP) 20 0 da ys 21 159 days 22 0 days 9 368 days 11 0 da ys 04/25/20004/25/200 04/26/20012/04/200 12/04/20012/04/200 05/05/19909/29/200 09/29/20009/29/200 Host Cutoff 311 days 311 days 311 da ys 311 da ys 311 days

Conduct Final Reqts Analysis to Cont ract Pr epar at ion Cont ract A war d for Integrate ATNSI RRI and ProATN DL AP Accept ed Design, I mplem ent , Te st ConfigurationFCA/PCA Complete CPDLC Build 1 and Complete ATNSI R RI, ASE Audits/Incorporate(ATNSI D LAP) 13174 days AS Es Host/ 06/01/19901/28/200 14 0 days 10 0 da ys 16 180 days Changes 18 0 days 15 50 da ys 278 days 01/30/20001/30/200 05/11/20005/11/200 07/24/20003/30/200 1723 da ys 05/02/20005/02/200 278 days 03/02/20005/10/200 255 days 203 days 04/02/20005/02/200 203 da ys 255 da ys 203 days

Test & Evaluation

Internal FAA Ho st PTR fixes, Regression Interoperability Testing & Host Sof tware SuitabilTesting, it y final Host Debugging 28 22 days Soft ware Uplevel 25 86 days 01/30/20002/28/200 29 40 days 10/02/20001/29/200 311 days 03/01/20004/25/200 311 da ys 311 days

System D eliveredInternal to T &E F AA Final Begin Integration Testing Interoperability Testing & (need Flight DeckIntegration inpu t, TestAcceptabilit ing y Testing DL AP PTR Fixes & OT&E Co mplete DLAP Site Regr essi on Test ing Debugging shows sim option)31 55 da ys 32 44 days 34 0 days 24 0 days 07/09/20009/21/200 09/24/20011/22/200 33 50 da ys 01/31/20001/31/200 05/02/20005/02/200 2640 da ys 30 0 days 11/23/20001/31/200 203 days 05/14/20007/06/200 07/06/20007/06/200 196 days 196 days 196 da ys 196 days 196 days 196 days

Implemen t at ion

Initial OperatingPrep. For Oper at ional RedinessDemonstration First OR D Capabilit y (IOC) 40 0 da ys (ORD) 42 0 da ys 05/15/20005/15/200 41 67 da ys 08/16/20008/16/200 05/16/20008/16/200 488 da ys 195 days 488 days

Site Prep, Host Implemen t at ion, Delivery to KeyField Site Familiarization SAT Testing Complete Inst allat io n & C heckou t , 37 10 da ys 38 63 da ys 39 0 days SAT Activities 02/04/20002/15/200 02/18/20005/15/200 05/15/20005/15/200 36133 da ys 195 da ys 195 days 195 da ys 08/01/20002/01/200 195 days

Human Factors

Demonstrate Human Factors Testing Design U sabilityDesign Stud y U sability Study Pilot HCI ResultController s HCI Results Procedures/Training Study AT SS/Pil ot /S yst em 51 85 da ys 52 0 da ys 49 87 days Complete 01/04/19904/30/199 01/04/1994704/30/19985 da ys 04/30/19904/30/199 04/30/19948 04/30/1990 days 07/01/19910/30/199 Integrat io n 580 days 57 86 days 0 da ys 0 days 247 days 20.4 days 1132 days 12/18/20004/16/200 04/16/20004/16/200 837 days 837 days Miami AT SS/Pilot/Syst em Integration Planning Flight Deck HCI EvaluationAnalize R esults Commun it y Consensus 56382 days 53 23 da ys 54 33 da ys 55 36 days 05/17/19910/31/200 08/25/19909/24/199 09/27/19911/10/199 11/12/19912/31/199 870 days 335 da ys 335 days 334 days

Review and Update Human F actors Requirements and Guidance 45 304 days 12/01/19901/28/200 1153 da ys

RRI/ASE Soft ware Delivery

CPDLC and CM A ASE's Delivery Pre-CTS Soft ware Delivery 60 0 da ys 61 0 days 02/07/20002/07/200 RRI/AS 22302/07/200 days02/07/200 221 days ARINC VDLM2/ATN for ATS + ACARS

Ground Based FAA-ARI(Service NC Contract for Critical Airports/EnRoute Sv c Lev el Ag ree m en t AR INC, FAATC G/G Service for Miami live CPDLC IRD/ICD Provider/FAA) NetMgtBld 1 IAGS Deployed AVLC Testing w/RCollins wi t h F A A interop @ FAATC Post-operations operations 69130 days Description/Reqs 65100 da ys 89 140 days 67 0 da ys 85 0 days 72 60 days 81 40 da ys 90543 days 05/28/19911/25/199 70 87 da ys 05/28/19905/28/199 12/31/19912/31/199 06/30/20011/16/200 03/12/20006/01/200 04/06/20005/31/200 06/03/20012/13/200 06/03/20006/30/200 833 days 05/28/19909/27/199 944 days 403 da ys 489 days 0 da ys 0 days 260 days 91 da ys 0 da ys

Ops Perf & Success Int erf ace bet wee n ARI NC , Install production Ground A/ G inter op test ing wit h AR INC, FAATC A/G Criteria FAATC Station f or testing R ock well, Ameri can fo r interop @ FAATC 71 175 days 74 60 days 79 60 da ys P2E 82170 da ys 80 107 days 05/28/19901/27/200 08/09/19910/29/199 E 09/08/20011/30/200 12/01/20004/30/200 07/06/20002/28/200 401 days 852 days 657 days 609 days 657 days

Implement Testbed GSs Demo test/prep Support Demonstration OT&E Support 196 days 75 60 days 76 77 170 days 83 170 days 08/09/19910/29/199 11/26/19908/25/200 08/28/20004/20/200 07/06/20002/28/200 852 days 833 da ys 833 days 66 da ys

VD L M o d e 2/ AT N AT S M 2/ A TN IO C Install VD L Mode 2/ATNPre-operations in Development 66 0 da ys M iami/op s serv ice s yst88 em130 days 64 741 da ys 04/30/20004/30/200 86 120 da ys 12/03/20005/31/200 06/29/19904/30/200 119 days 10/15/20003/29/200 0 da ys 119 days 0 da ys

American Airlines CPDLC Pr ogram

Data Link Implementation Review 92 156 da ys 06/01/19901/04/199 0 days

American 767 Pr ogram

Retrofit Plan for American Cockpit Integration H MI Procedure Refinement for FAA HMI Cockpit Survey Fleet program with T ech C e nt er 97 4 days PET AL IIe 11/09/19911/12/199 95 43 days 98131 days 109133 days 0 days 02/01/19903/31/199 04/01/19909/30/199 06/01/20012/04/200 1370 days 1239 days 333 days

Maintenance & MEL Manuals Complete 107485 days 10/26/19909/01/200 422 da ys

Aircraft and Operating Maintenance Crew Manuals Compl et e Training & Procedures for 106152 days PET AL IIe 1080 days 06/01/20012/31/200 12/31/20012/31/200 337 days 337 da ys

AA Approved Training Program 104 451 days 01/08/19909/29/200 402 da ys

Procedure/Training Sim Hard ware Sim Hardware Int egrat e A TN D ispla ys Development/R efinement Configuration 1 Configuration Final Sim Data Link Provision for Miami 101120 days and CMU in Sim 99219 days 10067 days 07/03/20012/15/200 102 46 da ys 110578 da ys 04/01/19902/01/200 03/01/20006/01/200 03/17/19906/01/200 295 days 04/30/20007/02/200 443 days 336 days 316 days 200 da ys

Rockwell CMU/VDR Development ( need ties to cert)

CMU/VDR Functional Avionics/Ground System CMU CPDLC/C M A Ar chitect ure/H MI CMU CP DLC/CM A Testing and Updates R/L CMU Available CMU Updates and Release Planned B/L CM U Avail Preliminary Design Definition Finalized Development Testing 0 da ys 15 wks for Cert 126 ( Est imat e d) 128 129 122129 days 121 0 days 123325 days 48 da ys 02/26/20002/26/200 10/08/20001/18/200 1300 days 01/01/19906/30/199 06/30/19906/30/199 09/28/20012/04/200 12760 days 01/18/20001/18/200 204 days 07/01/19909/27/200 204 days 12/05/20002/26/200 204 days 41.6 wks 208 days 204 da ys 204 da ys 204 days

R/L VDL Mode 2 VDR B/L VDL Mode 2 VDR VDL Mode 2 Development Available ATNSI RRI Delivered Av ailable 117295 da ys 1180 days 01/11/19902/25/200 1130 days 02/07/20002/07/200 114 0 da ys 357 days 10/01/19910/01/199 221 da ys 06/30/20006/30/200 571 days 376 days

CMU ACI CPDL C/CM A AC I ASE DeliveredASE Integ/DebugAT NSI R RI Inte g/Debu g 1240 da ys 119151 days 02/07/20002/07/200 125149 days 02/23/20009/20/200 02/15/20009/08/200 223 days 217 da ys 209 da ys

767 Fligh t Deck Int egr at ion Act iv it ies ( nee d t ies t o cer t ) Aircraft Configuration Provision Aircraft for Review TS A C om pleteCockpit Display Definition Black Label CM US 767' s Av ailable f or M ia mi 1340 da ys 13513 days 1420 days 132 64 da ys 06/11/19906/11/199 06/14/19906/30/199 14170 days 04/26/20004/26/200 01/01/19903/31/199 01/21/20004/26/200 0 da ys 201 days 204 days 208 da ys 208 da ys

Q uick Ch ange E C O' s ( t ie B767 Deliveries 767 to out of service testing) 133 67 da ys 02/01/19905/04/199 140153 days 0 da ys 07/03/20001/31/200 890 days

Development of Cockpit Provision Aircraft for Red VDL-2/CMU/Flight Deck Cockpit integration Display/W iring Updates Label avionics Integration Complete 136450 days 137140 days 13840 da ys 1390 days 06/14/19903/02/200 08/01/20002/12/200 03/05/20004/27/200 04/27/20004/27/200 200 days 214 days 200 da ys 200 days

FAA Flight Standards Tasks [Streeter, Don]

American Develops POI Checks Ops Approval Operat io nal Spe cificati on Flight Standards Data AG C Ap p r ov a l Public Comment AFS Team Approval AC 120.c om FinValidatio al n Test Plan Test Plan Validat io n Test Sp ec/Op e rat ion al Collection durin g live ops 14544 days 14623 days 14768 days 1480 days 15323 da ys Approval Issued (AFS POI) 04/15/19906/15/199 06/16/19907/16/199 07/19/19910/20/199 11/01/19911/01/199 151220 da ys 15265 da ys 06/05/20007/05/200 155360 days 11/01/19909/01/200 09/04/20012/01/200 1540 days 06/04/20010/20/200 557 days 557 da ys 557 days 550 da ys 419 days 11/01/20011/01/200 550 days 550 days 335 da ys 182 days

American develops and submits operational approval plan spec 150220 days 11/01/19909/01/200 638 da ys

FAA Certification Tasks (Steve Van Trees, AIR)

ACO approval of CMU/Flight Deck RTCA 172 VDL Mode 2 CMU/Flight DeckCMU/Flight Deck Detailed Installation Inspection AI R-100 T SO CMU/Flight Deck Detailedcert ification plan, design A A PET AL II Ma astr ich t PET AL II PET AL II Test Result Address PETAL II Test AF M Sup plemeFAA nts Type Design Preliminary Design (tie to AG C App rov al Public Co mmenComment t Disp ositionMOPS (Son Tran)AC 20DC Final Preliminar y DesignDesign (tie to (tied to C MU/flight FirstDeck Flight An al ysis Issues an d Fix Completed Ap prov al Rockwell/Boeing efforts) 168270 days Design C omplet e data completion) 18360 da ys 1621 day 163 45 da ys 16430 days 167296 days 165 0 days CompletedRockwell/Boeing efforts) 182 1 da y 18410 da ys 185 15 wks 1865 days 187 1 day 175360 da ys 04/15/19904/15/199 04/20/19906/21/199 06/22/19908/02/199 04/15/19906/01/200 08/03/19908/03/199 1760 days 177160 days 06/02/20006/14/200 08/28/20017808/28/2000 da ys 17980 da ys 181 23 da ys 05/31/20005/31/200 07/02/20009/21/200 09/24/20010/05/200 10/08/20001/18/200 01/21/20001/25/200 01/28/20001/28/200 09/01/19901/17/200 0 da ys 383 da ys 383 days 383 da ys 01/17/20001/17/200 01/18/20008/28/200 794 days 08/29/20012/18/200 04/30/20005/30/200 208 days 323 da ys 794 da ys 323 days 323 days 323 days 323 da ys 229 da ys 229 days 208 days 41.6 wks 632 days 632 da ys

American develops High Level Certification AI R- 100 T SO pr ocess AF M Sup pleme nt s certification plan and Ap plication 170143 days complian ce dat a Developed 172 0 da ys 04/15/19911/01/199 17490 days 173220 da ys 01/31/1990 da01/31/199 ys 1217 da ys 08/03/19906/05/200 96606/06/200 days10/09/200 383 days

Operational Environment Definition (FAA-Hritz) 158214 days 12/07/19909/30/199 560 days

Operational Saf et y As sessm ent ( F AA - Hrit z ) 159214 days 12/07/19909/30/199 560 da ys

Syst ems Requir ement s and Procedures 160167 days 05/03/19912/21/199 Petal II 502 days

Path at or near critical Key Events Fixed Events

Figure 2. Critical Path Analysis Example Technical Critical Path Limited References. The term was introduced to this paper by Dr. Yeo of the Nanyang Technological University during discussions about TPMs, February 19, 2009. It can be found in various references where it addresses the series of technology developments that are necessary for a development to be successful. One example is a briefing on planning for advancement in solar energy by the U.S. Department of Energy (2006), Figure 3.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 13

Figure 3. Technical Critical Path Reference The term can also be found in a Defence Acquisition University briefing on Systems Engineering Plans. Measuring this overall technology development would appear to be on the scale of Technology Readiness Levels. Technology Readiness Levels. NASA identified a nine-level scale to address the maturity of technologies for application to programs. As can be seen in figure 4 (Mankins, 1995), the levels can be broadly interpreted. The differences between levels are primarily driven by factors other than specified performance. For instance, a demonstration can be of a component or a subsystem. It can be performed in a laboratory environment or the appropriate space or ground environment. This is appropriate for addressing general progress in a longer term program over several phases. However, there is an opportunity to tie the levels of technical progress in a specific program to a more detailed definition of the progress.

Actual system “flight proven” through successful mission operations

Actual system completed and “flight qualified” through test and demonstration (Ground or Flight)

System prototype demonstration in a space Environment

System/subsystem model or prototype demonstration in a relevant environment (Ground or Space)

Component and/or breadboard validation in relevant Environment

Component and/or breadboard validation in laboratory environment

Analytical and experimental critical function and/or characteristic proof-of-concept

Technology concept and/or application formulated Basic principles observed and reported

Figure 4. Technology Readiness Levels

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 13 Technical Performance Measures Need for technical input. The primary emphasis of management systems is on cost and schedule. The Earned Value Measurement System (EVMS) is intended to connect the tracking of dollars with the actual progress being made. While the descriptions of tasks in a Work Breakdown Structure (WBS) call for a definition of technical objectives, specific guidance on how to establish these objectives is not readily available. This leaves the integration of the EV management techniques and the technical effort to the reader. In practice, the WBS tasks often refer to the accomplishment of a task or completion of a product without a qualitative description of what the task or product is intended to achieve. As a result, the EVMS and technical progress tracking are generally addressed in a disconnected fashion. Properly constructed and implemented TPMs can provide the technical progress information to address the technical critical path and help the integration of the technical and management disciplines. However, several common mistakes in TPM use must be avoided to allow this integration to take effect. Why more on TPMs. Descriptions of TPMs and how to use them abound. For instance, the INCOSE Systems Engineering Handbook (INCOSE, 2006) has a good description of what they are and an example, figure 5. They have been used in programs effectively for decades. Therefore, it was with surprise that they were observed in systems engineering training classes as the one concept that was most frequently problematic in exercises. Later, as students used their internal process assets in class to develop TPMs it was discovered that the processes did not address the key features of TPMs in their directions Figure 5. SE Handbook TPM Example or templates. In another instance, the same was found to be the case in an appraisal. What to measure. The first concern is to determine the parameters that should be measured. All measurement decisions of this kind should begin with determining the question to be answered by the data. In the case of TPMs, the status of the progress on the most critical design parameters is what the data should reveal. The determination of which parameters are the most critical is accomplished through a combination of requirements decomposition and risk management. Requirements decomposition. There are many terms for the highest level of requirements for a system. Measure of Effectiveness (MOE) will be used here but other terms with similar meanings include Key Performance Parameter, Most Important Requirement, and Key System Attributes. For instance, you might be concerned about cost of ownership for a vehicle. Gas mileage is one contributor. However, if that is achieved through a very complex control system that is expensive to build and maintain, the total cost, the MOE, may not be achieved. However, if the mileage is determined to be the critical component of the MOE and at risk, it would be a very likely candidate. Further, we would look for a detail parameter in engine design that is both the design challenge and a driving factor in achieving the desired performance. This trace of MOE to TPM is shown in figure 6 as described in a PSM and INCOSE report (PSM/INCOSE 2005).

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 13

Increasing Measures of Technical Resolution Effectiveness & Periodic Insight (MOEs)

Key Measures of Performance Performance Parameters Mission (MOPs) Technical Needs or (KPPs) Technical Insight Operating (Progress Performance Issues Increasing & Risk) Scope of Measures Technical Solution (TPMs)

Technical Measures are Interdependent

Figure 6. Relationship between MOEs and TPMs Risk. Of course, the parameter in question should also show up on the list of technical risks as one of the higher risks in combined impact 1 and likelihood of occurring, as shown in figure 2 ty 7. TPM tracking then becomes one of the il mitigations as a monitoring technique. It also b a 4 provides the trigger for future decisions to add b 3 or cancel specific mitigation activities. Should ro 5 the progress not be satisfactory, additional P mitigations may be triggered or decisions for alternate designs made. Should progress exceed plans, efforts on back-up designs might Severity be canceled. Risk #1: Related to selected TPM Plan for changes. The next step is to define the projected change in the monitored parameter Figure 7. TPMs and Risk over the life of the development, figure 8. As we have been discussing it, this would likely be an improvement to meet the desired value. However, there may be a positive margin at the start of design, e.g., the product looks to be well under a weight limit, and we want to make sure that the margin does not disappear in the first few weeks of serious design. In figure 5, that case is represented by the term “limit” rather than “target” for a performance objective we are trying to achieve. In the case where the product starts out overweight, the initial prediction would be above the target and the prediction curve would show the plan for reduction.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 13

Target or Limit Reserve margin not shown How will we close (or maintain) Expected this gap? change

Initial estimate

Time

Figure 8. Prediction of Change The shape used in this example is a simple exponential form where more rapid progress is expected early in the development with progress becoming more difficult as the program proceeds. This is a notional example. Planned progress for a specific program may look considerably different. For instance, figure 5 shows a TPM where a design margin is monitored and the plan is to minimize degradation early and expect some larger changes later in the program. Some examples to avoid are presented later in this paper. Adding ties to events. The next step is one that is most commonly omitted. This is essential to the integration or separation of the technical and management processes. In this step, progress on the plan curve is explicitly tied to specific events in the WBS and schedule, particularly the events on the Technical Critical Path, as shown in figure 9. One description of tasks says to “identify physical products, milestones, technical performance goals, or other indicators that will be used to measure progress.” (NASA) In practice, this means referring an early design tasks or prototype effort to the final target specification. The results are misleading if the expectation is that the final number will not be met at that time or that a positive margin will be mostly retained. It is more realistic to make the technical performance goal for that early event to the predictions in the TPM plan. If the top highlighted task in the WBS in figure 5 is a prototype development to address the technical risk relating to the TPM parameter, the technical goal for that task should be the same as the value on the TPM plan curve at that time in the program. In this case, it appears to be about half way between the initial estimate and the final target.

Target or Limit Estimate results at WBS

each task xxxxxx xxxxxxxxxx Predicted xxxxx xxxxxxxxxxxx completion xxxxx xxxxxxxxxx Progress Xxxxx xxxxxxxxxxxx xxxxx xxxxxxxxx Project Schedule xxxxx xxxxxxxx and map to xxxxx xxxxx Initial xxxxx xxxxxxxxxx graph xxxxx xxxxxxxx estimate xxxxx xxxxxxxxxxxxx xxxxx xxxxxxxx xxxxx xxxxxxxxxxx xxxxx xxxxxxxxx xxxxxx xxxxxxxx xxxxx xxxxxxxx xxxxx xxxxxxxxx

Figure 9. Integration of TPM and WBS Measurement method. In conjunction with tying to the WBS, the measurement method for each critical point in the schedule, figure 10. The method should be integrated into the WBS. It may be

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 13

planned that the estimate of weight at the completion of initial design drawings will be by analysis rather than actual measurement of a prototype. If so, that point needs to be clearly defined at the outset, not debated at the completion of the design task.

Target or Limit

Defined updates •Analysis Should connect Initial • Prototype to TEMP, other estimate •Test verification •Etc. plans, and design reviews

Time Figure 10. Identifying Measurement Methods Additional integrations. In some cases, particularly later in the development, the TPM plan should be tied to the Test and Evaluation Master Plan and other verification plans. A mismatch between the pass/fail criteria and expected progress at the time of test will certainly be problematic. Another relationship to maintain is with the entrance and exit criteria for design reviews. If the plan was to show 90% performance on a challenging parameter at CDR as demonstrated by a prototype, that is what should be expected at the review. The customer, internal or external, should not be complaining that it isn’t 95%. Likewise, if the results are only supported by analysis because the prototype is not completed, the designer should expect a strong response. In either case, neither side should be caught off guard and have to argue adequacy of method or results at this point since the plan was agreed to at the start. Thresholds. There are several approaches to Thresholds. Figure 11 depicts an upper and lower bound. This is the range in which results are acceptable without further action. If performance deviates beyond these limits, management action is to be taken. If the performance deviates in the negative direction, additional resources may be applied or additional mitigation actions may be triggered. As long as the performance is within the bounds, internal and external customers and management should not consider it a problem. In another approach, double boundaries are applied with green, yellow, and red bands around the target. In this method, green is not a problem, yellow is handled within the program, and red is addressed by higher management outside the program.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 13

Target or Limit

Variance Thresholds

Time Figure 11. Addition of Thresholds Some organizations do not use a “good” side limit and some use the plan as the boundary. Figure 5 is an example of this approach. While the argument is made that there is no value in investigating good news, there are two reasons why this can be appropriate. One is to verify that the good news is real. The other is to evaluate the possibility of reallocating resources to more needy areas. While the good side is not as critical as a negative threshold breach, it is worth more that casual attention. Tracking Progress. Of course, the efforts so far are without value if the actual results are not tracked as the program progresses, figure 12, and corrective action taken as necessary. The tracking frequency is established in the original plan along with the collection and analysis methods. Relationships to other risk mitigations are maintained in the risk plans.

Target or Limit

ce n a rm o rf e Plan P Actual Start Time Delivery Figure 12. Tracking Actuals Integration When the complete process is followed, TPMs and risk management combine to add the technical planning and progress to the overall management process as shown in Figure 13 and real meaning to tracking the Technical Critical Path.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 13

EAC Budget Baseline Management Reserve

PM Baseline $$ Plan WBS Actuals SV CV xxxxxx xxxxxxxxxx Cost xxxxx xxxxxxxxxxxx xxxxx xxxxxxxxxx EV Xxxxx xxxxxxxxxxxx xxxxx xxxxxxxxx Time xxxxx xxxxxxxx xxxxx xxxxx 1 xxxxx xxxxxxxxxx 2 Ta rg e t t xxxxx xxxxxxxx n xxxxx xxxxxxxxxxxxx e xxxxx xxxxxxxx 4 l xxxxx xxxxxxxxxxx 3 e Schedulexxxxx xxxxxxxxx 5 p Event xxxxxxJANFEB xxxxxxxxMARAPRMAYJUNJULAUGSEPOCTNOVDEC o xxxxx xxxxxxxx r Milestonesxxxxx xxxxxxxxxAWD PDR CDR TRR FCA P HW Design /PCA HW Fab HW Test Schedule Technical SW Design Time SW Test/Int HW/SW Integ System Test

Figure 13. TPMs integrating SE and PM EVMS impacts. There are some complications in using this integrated approach. When a task is completed but the performance is not within the planned value limits, the earned value should not be taken. As with any incomplete task, the impact on downstream tasks must be specifically addressed. The complication comes in the case of a prototype, for instance, where the course of action is not to continue to redo the prototype. Even though the original task, building and testing the prototype, will end, care must be taken to properly handle whatever the additional work results from the failure and not just take full earned value and move on. An easier to accept but still less than straightforward situation exists if the activity exceeds the limits on the positive side. The concern here is to assess reductions in future tasks that may be available. The most complex situation would be where several requirements are in play and some go to the good side while others go to the bad. In this case, a decision based on overall risk exposure from the combined factors is a more reasonable way to address the question. A net risk reduction goal could be identified as the objective rather than individual parameter changes. TPM Uses The initial value of TPMs is to proactively forecast the technical progress to be achieved instead of only reactively responding to technical events. TPMs identify differences between actual versus planned performance to provide informed management. This enables the technical and program management to assess and predict progress towards achieving the performance values, determine the impact of these differences on system effectiveness, and react effectively. TPMs provide early indicator of risks and problems requiring management attention. Figure 14 shows the use of TPMs in several modes. On the left, the TPMs are monitoring performance gaps as they are closed. On the right, the TPMs monitor positive margins to assure that they do not disappear too rapidly. In the center is an alternative that may not be used often, if ever, but should be considered. If a key parameter is initially on target, but has a history of varying making it a risk, it would be a candidate to monitor.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 13

Manage Improvement Manage Margin f Ta rg e t o n Limit th ty o li ti id i c Maintain w e b e d s a t n U b e Level a ro D B P Time Time F B Target or T M Limit g t in u s p s e Time h e m g c i u Target o T Limit ro r h P T Time Time Figure 14. TPM Use Examples In addition to the tracking progress, TPMs also allow the manager to determine where opportunities exist to make design trades to reduce overall risks. One example of this was described by Hamman and Mackertich (INCOSE 2007) where the understanding of positive technical margins enabled a trade of technical performance to reduce cost with significant savings. TPMs can also be used to monitor performance improvement after delivery as shown in figure 15. However, this is an additional use and not the main focus of predicting technical progress during development.

) F Extended Target B T Initial Target M ( ty il b Delivery lia e R

Time

Figure 15. TPM Use After Delivery Things to Avoid Driving through the rear view mirror. Some programs and organizational processes have fallen into the trap of defining TPMs as reporting past results only as in figure 16. In addition to the obvious issue that the future expectations are not defined, the current state has no target to define its acceptability. In this case, there is no decision point to determine when corrective action is needed.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 13

Today t h Ta rg e t ig e W

Program Schedule Time

Figure 16. Looking in the Rear View Mirror Straight line predictions. This mistake, figure 17, is not acceptable in any progress prediction with extremely rare exceptions. The last few improvements are almost always a bit slower. However, the practice continues to occur.

t h ig e W Ta rge t

Program Schedule Time

Figure 17. Straight Line Prediction The miracle. One guide that had some good guidance on TPMs seems, unfortunately, to have fallen victim to the elimination of military standards. While the text was valuable, a different kind of lesson can be gleaned from the example at the end of the guide, figure 18. In this example, progress on throughput time in information processing software was to be tracked. The time per message needed to be reduced to meet the requirements. Modest progress was predicted through the development process. Some degradation was predicted at system test. However, the miracle predicted was the sudden improvement when the system is to be fielded and live data added. In reality, it should be expected that things get worse when real world is encountered. One possible explanation is that the chart meant track throughput and to reflect a positive margin with realistic expectations of the margin disappearing when fielded.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 12 of 13

e System test im T g in s Unit test s e c ro Target P Live data test

Program Schedule Time

- USAF TPM Guide Example Figure 18. Predicting a Miracle Only used after delivery. The use of TPMs after delivery is an acceptable as described above. However, one error that has occurred frequently in classroom exercises is to believe that TPMs only start at delivery, figure 19. The key is to predict and track performance throughout development.

) F B Extended Target T M Initial Target ( ty il b lia e Delivery R Now Time Figure 19. TPMs after delivery Summary The concept of a Technical Critical Path is evident in some form on most projects that depend on technological advances. In order to make it work, there needs to be a measurement approach that will yield actionable information with enough granularity to allow for management response. Technical Performance Measures are a basic part of technical management that can be applied to provide a that level of measurement to the Technical Critical path. While the concept of TPMs is quite simple, the execution has been sporadic in many programs and organizations. Following a few simple guidelines can produce effective measurement of technical progress and aid the management processes such as earned value measurement. If done correctly, the TPMs can be a missing link between the technical and management sides of a development project, provide

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 13 of 13

tracking of the Technical Critical Path, and increase the likelihood of project success. References INCOSE 2006, Systems Engineering Handbook - A Guide for System Life Cycle Processes and Activities, Version 3, Edited by Cecilia Haskins. Hamman and Mackertich, Get Smart – Enabling Enterprise Systems Intelligence and Decision Making Through Critical Parameter Management, INCOSE, 2007 Mankins, J. C. 1995. Technology Readiness Levels. White paper, NASA Office of Space Access and Technology. http://www.hq.nasa.gov/office/codeq/trl/trl.pdf. Office of the Deputy Under Secretary of Defense for Acquisition and Technology, Systems and Software Engineering, Enterprise Development. Systems Engineering Plan Preparation Guide, Version 2.01. Washington, DC: ODUSD(A&T)SSE/ED, 2008. Roedler and Jones, Technical Measurement, PSM/INCOSE 2005 Solar America Initiative: Technology Pathway Partnerships (TPP’s), US Department of Energy presentation for April 18-19, 2006 meeting in Chicago Systems Engineering Principles, Defense Acquisition University presentation, 22 May, 2008 BIOGRAPHY Jim Armstrong has practiced systems engineering for 40 years, performing various roles including configuration management, test, deployment, chief engineer, program manager, and program element monitor. For the last 20 years, he taught, consulted, and appraised systems engineering in industry and government. Also, he was on the author teams for most of the systems engineering standards and models.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 18

Becoming a Systems Engineer

Roy de Souza Gary Langford Naval Postgraduate School Naval Postgraduate School Naval Postgraduate School 222 Bullard Hall 1 University Circle 777 Dyer Road Monterey, CA 93943 Monterey, CA 93943 [email protected] [email protected] Copyright © 2009 by Gary Langford and Roy de Souza. Published and used by APCOSE with permission.

Abstract. Performance of systems engineering tasks requires appropriate skills and methods often surmised from marketing and advertisement pieces. What is the correlation between the ‘required’ skill set and the individual’s experience? We observe that skills seem to vest with time and that the systems engineer likewise becomes more proficient in those traits that improve project performance. Can these proficiencies be tied to time or years of experience? We apply a time-based exponential learning curve to data collected from three sources: classified job advertisements, surveys of practicing systems engineers, and pertinent literature. From a triangulation methodology we investigate the relationship between the number of years of experience and the breadth and depth of knowledge expected in the workplace. We then relate the contribution to work made by systems engineers to the number of their years of their experience. For the processes of requirement’s analysis and project management skills, systems engineers with 15 years of experience are twice as effective as those with 10 years of experience (with a 3.6 year error).

Introduction

Who should be hired to fill a job that requires systems engineering skills? Should the individual have 3-5, 8-10, or 15+ years experience? As Systems Engineers progress through their career they acquire and improve the skills and traits required by engineering solutions. Hiring practices, job assignments, acquisition of skills, and improvements in proficiencies are seemingly correlated to this progression. The authors outline an employment lifecycle for Systems Engineers which reflect characteristics that can occur within phases of their work, e.g., by years of experience, job title, annual salary (Cosgrove 2006). As defined by the essential skills and proficiencies found in the workplace, the systems engineer can be seen to ‘progress’ from a noviciate to a senior-level. Acquiring skills can be thought of as an economic transaction, whereby an employer pays an employee to learn. There can be two direct consequences of that learning: (1) the employer and the employee benefit (internalities), and (2) the customer benefits (externalities). Since the customer is not a direct party to the learning, but yet benefits, an externality is created. The learning of Systems Engineering skills can create internalities and externalities that can then be defined in terms of a profession and a discipline that apply holism to understand and solve

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 18

problems. Both internalities and externalities need to be observed and measured. We posit that the maturation of a Systems Engineers seems to be the consequence of a minimum of training, education, internalization and reflection, experience, and awareness. But, since the artifacts and their proportional contributions are as yet undetermined, this research frames an approach to identifying the artifacts that seem to play a role in determining a maturity curve for systems engineers. If the end state for a project is to satisfy stakeholder requirements within the allocated time and budget, then there would seem to be a set of traits that must be acquired or innately bred. While the data used in this research was particularly focused on Systems Engineering, we view our methodology, analysis, and measures as broadly applicable to disparate occupations and various avocations. We applied a qualitative methodology by (i.e., surveying practicing Systems Engineers, referencing current literature for defining their traits, and reviewing recruitment advertisements placed on the internet, and in magazines and newspapers. Data gathering was determined to be complete once a noticeable pattern evolved in each of the three data sets. This was consanguineous to performing a meta-analysis, less the presence of any statistical comparison (de Souza 2008). The check for independence between the data sets was non-exhaustive, but the coupling was assumed to be weak for the purposes of this research. Future work will test this assumption. Two variables were analyzed – years of experience and annual income. Scaling parameters were determined and plotted on a fuzzy logic scale based on triangulation comparisons between the three sets of data (Silverman 2005). Triangulation is a sampling strategy used to validate data (Denzin and Lincoln 1994). It is the focus of triangulation to determine the efficacy of a data source with regards to the dependencies of the intended consequences. Specifically, we applied the triangulation method of Frank, Frampton, and Di Carlo, 2007 that advocates establishing categories of repeated elements. Through corroboration of these elements by multiple occurrences, the data collection can be declared “steady state” and then terminated. The three sources were analyzed to determine these categories of common elements. The error of the effect size normally associated with most meta-analyses was omitted, as no statistical analysis of the data was required. The Fuzzy Logic model defined by Zedah in 1965, was applied to the learning curve to compare the traits of System Engineers and correlated to their years of experience, then later with comparison to annual salaries. The fuzzy logic scale offered a convenient and substantiated method of describing the patterns of data. A fuzzy set is class of objects with a continuum of grades with varying degrees of membership (Pedrycz and Gomide 2007). By combining the categories, grades (scaling factors), and associations (membership), a series of fuzzy logic scales were developed and used to accommodate and plot the three data sets. Over seven hundred traits were gathered and correlated from the three data sources. A Fuzzy Logic Scale was created to summarize the traits desired by employers. The scale was designed in accordance with the number of years of experience and was scaled from 1 to 10, with 1 being the least experienced and 10 being the most experienced. The fuzzy scale was designed to represent high numbers of correlations of traits between data sets with a high marker and a low marker for small numbers of correlations. Traits were binned according to membership relations on a linearly increasing scale for years of experience. Figures 1, 2, and 3 summarize the data sets for surveys, classified advertisements, and literature review, respectively. An attempt was made to eliminate potential bias in either the logic of the fuzzy

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 18

scale or in the binning of data by comparing the traits obtained from the literature review and the survey. A key observation of the amalgamated data sets was the grouping of certain traits at various levels of systems engineering experience in the maturity cycle. To further analyze this pattern, learning curves were plotted and the relationship determined between the power and coefficient of the curves relative to a zero starting point. This zero-point normalization assumes no prior knowledge of the traits as so distinguished.

We note two limitations in this paper and presentation is the restriction to literature only gathered in the past forty years. The early years of writings on Systems Engineering are quite productive and competent with requisite skills and proficiencies. A broader, more comprehensive perspective can be found in de Souza’s Master’s thesis (de Souza 2008). Secondly, the data was collected only to include up to 15-years experience. For some of us, such a limited scope denies the true worth of longevity and maturity.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 18

Figure 1. Fuzzy Scale of Amalgamated List (Survey)

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 18

Figure 2. Fuzzy Scale of Amalgamated List (Classified Ads)

The requirements of a Systems Engineer as printed in the classified ads were chosen to allow the data to be ‘naturally occurring’ (Silverman, 2005) without interventions or special setup of interviews or specific environments. However, it was not assumed the originator of the advertisement was well versed and could articulate and present their needs and requirements. Accordingly, there is error in crafting the desired Systems Engineer requirements due to differences in understanding between the hiring manager and the actual needs, the number of words or appropriate symbols that can be compressed into decipherable text, in addition to the restrictions (e.g., competitive disclosures) that need be placed on what can be stated about the traits desired. These ads were confined to those available on the Internet. Forty classified ads were used. Consequently, the newly hired Systems Engineer may possibly not have the requisite full range of desired traits.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 18

Figure 3. Fuzzy Scale of Amalgamated List (Literature Review) To review and analyze the literature a Meta-Visualizing Tool was developed (Figure 4) to show the relationship and relevance between the major sources of literature and the traits of systems engineers. Table 1. Areas of Concern for Classified Ads Area of Concern Details Implications The skills of the potential Inaccurate requirements 1 Need of Organization hires may be misunderstood specified in ads Interpretation of The requirements statements Inaccurate requirements 2 Needs / may not cover the needs specified in ads Requirements Limiting Limited ad space results in Inaccurate requirements 3 Requirements only the basic requirements specified in ads Ability matching to The applicants may not match May not be the most 4 Classified Ads their abilities to the real needs suitable candidate hired

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 18

Hall 1962

Figure 4. Meta-Visualizing Tool for Literature Review Thus a compilation of articles and books that covered the traits of successful Systems Engineers spanned 40 years. The ability to think systemically was seen essential (Baird 1971); Frank Zwikae, and Boasson 1997); Frank 2000); Augustine 2000); Frank and Elata 2005) and Davidz 2005). Further insights resulted from investigations into the roles of Systems Engineer (Sheard 1996), a review of specialized Systems Engineers, in Space Systems (Moore 2000), and in comparison studies with Systems and IT Architects (Frank, Frampton and Di Carlo 2007). A triangular comparison (0) was performed on traits gleaned from reviewing published literature, those displayed in online classified ads, and responses from surveys completed by practicing Systems Engineers.

Traits from Literature Review

Traits of a Systems Traits from Engineer Traits from Survey Classified Ads

Figure 5.Triangular Comparison

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 18

Attention then focused on acquisition stages for to acquire these traits. In the process of understanding the development of systems engineering skills, the essential elements of systems engineering were compiled. We chronicled the range and attributes of systems engineering over forty years beginning in the 1960s (Figures 6a (1960s to 1980s), 6b (1990s), and 6c (2000s), as the discipline matured and explored its boundaries.

Class of Systems Application of Engineering Efforts Analysis Compatibility Application of Scientific Efforts Broad Technical Plan Current Engineering Better Design of man-organized and man- Business made systems Development Planning Designing of Systems Comprehending Complex Systems Development Studies Formulate Economic Objectives Integration Process Exploratory Planning Formulates Operational Interdisciplinary Approach Performance Integrated Approach Maintenance Analysis Modeling Integration Process Operation Analysis Research Iterative Process Scientific Methods Simulation Optimization through balancing objectives System Functional Analysis Program Planning System Reliability Analysis Recognition of System Systems Approach Objectives

Systems Engineering Systems Task Analysis Descriptive Phrases Phrases of Descriptive System Approach Use of an Iterative Process System Considerations System Studies Te s t i n g

1970s 1980s 1960s Years Figure 6(a). Attributes of Systems Engineering (1960s to 1980s)

Evolve and verify Academic Discipline Planning Solving complex System Problems Analysis For Large Scale and Scope Policy Making System Analysis Analytical Formulation Post Implementation Balance of all System elements Formulation, Analysis & Assessment System Definition Interpretation of proposed Balanced set of system people Process Solutions System Management Identification of System Goals Procedure Balanced set of system product Production of Systems System Performance Information and Knowledge Professional Discipline Collaborative Approach Organization & Management Parameters Identification Qualitative formulation Combination of methods & tools Integration of the '-ilities' System Synthesis thru use of suitable Quantification of System Technical Effort related to methodological process Integration Process Goals Life cycle of product / Intellectual Discipline Comprehensive Approach Quantification of Systems system Interdisciplinary Approach Cost Assessment Quantitative formulation Technical Process Interpretation Creation of Alternatives Requirements Satisfaction Te s t i n g Iterative Process Process Creation of Systems Transform Operational Systems Engineering Systems Descriptive Phrases Phrases of Descriptive Decision Making Life-cycle View Resource Deployment Need to System design Designing of Systems Maintenance of Systems Resource Management Translation to Work Breakdown structures Development of information Management of System Risk Assessment Configuration Verification of design Document Requirements in Robust Approach Verification Process Specifications Management Process Satisfaction of Needs Engineering effort Encompassing the entire Management Technology Satisfy Customer Needs technical effort Evaluation Needs Satisfaction Scientific effort

1990s Years Figure 6(b). Attributes of Systems Engineering (1990s)

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 18

Balance of all System elements Parallel Process Complex Systems Requirements Satisfaction Conformance to Schedule Process Cost Effectiveness Methods Resource Allocation Creating and Executing an Satisfaction of Needs Interdisciplinary Process System Analysis Customer and Stakeholders System as a Whole consideration System Considerations Design Synthesis System Life Cycle Consideration Engineering Discipline Systems Thinking Engineering of Complex Systems Team Approach Evaluation Testing Functional Analysis To G u i d e Initial definition of system Systems Engineering Systems Descriptive Phrases of Descriptive Top- Down Approach Interdisciplinary Approach Total Operation Iterative Process Verification Process Life-cycle View Needs Satisfaction Non-Sequential Process Non-Traditional Engineering discipline

2000s Years Figure 6(c). Attributes of Systems Engineering (2000s)

Methodology. Data Gathering involved a series of processes as depicted in Error! Reference source not found.. The sources of the data, i.e., literature review, surveys and classified ads, were assumed to be representative of the Systems Engineering community. To determine the accuracy and precision of the data, collection, and analysis for the survey, errors were hypothesized, listed, and managed. Some of the inherent survey errors were gleaned from Demming (1994) in Denzin (1989), and then adapted to improve triangulation methodology. The survey of practicing Systems Engineers allowed the thesis to be initiated in two ways. The surveys allowed the gathering of the traits that make a good Systems Engineer that in turn facilitated an initial categorization from which to acquire traits through classified ads and the literature review. The methodology stipulated that the data acquired be in its natural state of existence, i.e., without intervention or coercion from the surveyors, the surveying instrument, or the enterprise and its framework of operations and activities. The natural state of existence refers to the absence of possible biasness, i.e. skewedness in survey questions, propaganda in classified ads, etc. Hence, acquiring data from the classified ads and from the literature review was deemed to be in the natural state, as is it taken that there are no ‘interference’ in the contained information. Three sampling strategies were considered: Theoretical Sampling,1 Illustrative Sampling,2 and Triangulation. The aim was to match the sampling strategy with the style and ilk of the Systems

1 Selecting groups or categories to study on the basis of their relevance to the research question and theoretical position (Mason 2006)

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 18

Engineering community. The objective was to identify and match traits that were ideally and pragmatically representative of both reality and expectations. Triangulation (Denzin and Lincoln 1994) was determined to best determine whether a specific source of data would be robust to changes in methods and local and individual sensitivities to the field. Hence, the combination of multiple methods, empirical materials, perspective and observers in a single study is best understood as a strategy that adds rigor, breadth, and depth to any investigation (Denzin and Lincoln 1994); (Flick 1992). Triangulation can be understood from five definitions – four offered by Denzin in 1978: Data,3 Investigator,4 Theory5 and Methodological6 Triangulation; and a fifth (Janesick 1994): Interdisciplinary7 Triangulation. Figure 8 illustrates triangulation.

Analysis of Data. A‘modified’ meta-analysis methodology was used to analyze the data. Data mining dealt with structured databases of facts (Hearst 2003), while the text mining uncovered trends and patterns in literature and classified advertisements. A ‘modified’ meta-analysis methodology was used to analyze the data. Data mining dealt with structured databases of facts (Hearst 2003), while the text mining uncovered trends and patterns in literature and classified advertisements.

Selection of Sampling Strategy Gathering Data Sources

• Identification of • Triangulation (Mason, •Survey Sources 2006) (Denzin, 1978) • Classified Ads • Limitations of (Denzin and Lincoln, • Literature Reviews Sources 1994)(Janesick, 1994) • Usage of Sources ¾ Data Triangulation ¾ Investigator Triangulation ¾ Theory Triangulation ¾ Methodological Triangulation ¾ Interdisciplinary Triangulation • Analytical Induction (Mason, 2006) (Denzin, 1989) Figure 8. Data Gathering Process

2 Relationship between sampled contexts and phenomenon and the population of interest is illustrative/ evocative in nature. This approach seeks only to provide a ‘flavor’ to the population (Mason 2006) 3 Use of a variety of data sources in a study (Denzin and Lincoln 1994; Janesick 1994) 4 Use of several different researchers or evaluators (Denzin and Lincoln 1994; Janesick 1994) 5 Use of multiple perspectives to interpret a single set of data (Denzin and Lincoln, 1994; Janesick 1994) 6 Use of multiple methods to study a single problem (Denzin and Lincoln 1994; Janesick 1994) 7 Consideration of other disciplines to study a single problem (Denzin and Lincoln 1994; Janesick 1994)

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 18

Fuzzy Logic. An amalgamation of over 700 traits was completed along with the counts for each trait. A Fuzzy Logic Scale was created to summarize the traits desired by employers. The scale was designed in accordance with the number of years of experience and was scaled from 1 to 10, with 1 being the least experienced and 10 being the most experienced. The original idea behind fuzzy logic, as advanced by Zedah in the mid-1960s, was to initiate and propagate the transition from traditional mathematical modeling in engineering to a new, much more qualitative, ‘rough’ modeling using fuzzy sets and fuzzy methods (Bandemer and Gottwald 1995). Employers desire all Systems Engineers to possess good oral and written communications skills, coupled with a team player spirit. For values of 3+ on the fuzzy scale of experience, Interpersonal Skills, Analytical Skills, Systems Thinking are sought. To eliminate bias in the fuzzy logic experiential scale built for classified ads, an amalgamated list of traits obtained from analyzing literature and surveyed was compared to the fuzzy logic experiential scale for classified ads (Figure 9). The top 3-traits listed in Figure 9 are satisfying requirements, cost management, and project management. Following the retention percentages by FitzGerald (2005), we assume that the experts (those with above 15 years of experience) are able to retain 75% of the essential traits listed for a successful project. Experts seem to acquire traits from involvement in many projects. These experts have learned their trade, and are able to contribute all of the 75% that they have retained. Based on the listed traits, initial contributions of a novice Systems Engineer appear to be 15%.

Total Systems Consideration / Production

Evaluation of Technical Plans / Evaluation of Systems

Engineering Analysis

Cost Management Product Development Process / Coaching Skills Matrix relationship understanding

Coordinating Skills Mentoring management staff

Quality Management

Functional Definition Modeling / Interface Management

IT Skills / Software Experience

Technical Skills / Attention to Details Documentation / System Architecture

Subject Matter Expert in an Engineering / Technical discipline / Trade Studies / Human Systems Integration

Decision Making Skills / Policy Making / Certification

Manage Teams / Organization Skills / Standards Familiarization

Contract Knowledge / Problem Solving Skills / System Design / Adaptable / Computer skills with SE Tools

Systems Thinking / Testing Skills

Analytical Skills / Handle Product Design Specifications / Leader / Multi Tasking

Verification and validation / Complex Systems Handling / Interdisciplinary Knowledge / Project Management Skills / Specific Domain Knowledge

Customer Management / Focus

Integration (Function, Systems, Technical) / Systems Engineering Process

Interpersonal Skills / Requirements Analysis / Risk Analysis Oral / Written Communication Skills / Team Player

1 3 5 7 10 Low Mid-Low Mid Mid-High High 1 year 2 - 3 years5-7 years 10 years >15 years Figure 9. Fuzzy Logic of Traits Correlated with Years of Experience (Classified Ads) Raccoon (1996) covered fifteen forms of the learning equation. As different sets of traits appear to be acquired over the years of experience of performing Systems Engineering, we adopted the time-phased approach to plot the acquisition of traits. Hence, the exponential learning curve is

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 12 of 18

adopted. Merging retention (FitzGerald 2005) with essential traits for success, then coupling the scales to which traits are tagged (Figure 9) shows the following relationship (Figure 10).

Learning Curve of Traits

0.80

0.70

Requirements Analysis and 0.60 Project management Skills Learning Curve 0.50 y = 0.1171e0.1238x 0.40 Cost Management Learning 0.30 Curve

0.20 0.3219x

ContributionProject to y = 0.006e 0.10

0.00 02468101214 Year of Experience

Figure 10. Relationship between a Systems Engineer Contributions and Experience Conclusion

This research applied sociological research methodologies to define a maturity curve for Systems Engineers. By developing a fuzzy scale from which to compare data garnered from literature, recruitment advertisements, and surveys of practicing Systems Engineers, an exponential relationship was applied to contributions to project success relative to the number of years of experience. The results indicate that skills in requirements analysis for Systems Engineers with 15-years of experience are three-fold better than Systems Engineers with 6-years of experience. A summary of traits over the 40-year time frame (1960s to 2000s) is shown in Figure 11. The growth in the number of traits over time is due in part to the acknowledgement that the purview of Systems Engineering has increased, as well as to continued refinement of the work and required skills.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 13 of 18

Problem Solving Skills Abstract Thinking Project Management Analysis of Alternatives Quality Assurance Architectural design Requirements Analysis Knowing when Asking the right to Stop Resource Allocation questions Leader Risk Analysis Good Technical Requirements Change Manager Planning Owner Learning By Risk Management Complex Systems Experience Technology Design (Systems) Understanding Simulation Holistic View Awareness Maintaining Analysis (Systems) Concept Design of Things Design Integrity Software Engineering Interdisciplinary Concept generation System Knowledge Verification and Management Solutioning validation Thinking Controlling Skills System Architecture Systems Logistics/Ops Mentoring Skills Interaction Creative Systems concepts Objective Engineer System Planning Critical Thinking Modeling Systems Engineering Leader Technical Glue for Process Analysis Projects Decision Making Skills Monitoring Team Player Design (Logical) Multi Systems Synergy Simulation Customer Interface Disciplined Multi Design (Physical) Systems Thinking Disciplined Systems Technical Manager Negotiating Engineering Design (Systems) Team Leader Information Skills Process Team Player Manager Design Process Objective Analysis of Functional analysis Technical Knowledge Alternatives Process Engineer Optimization TraitsEngineer of a Systems Testing Generalist Oral Good Engineering Coordinator Holistic View of Things Communication Trade Studies Judgment Skills Integration Understanding of Requirements Organizational Relationship Analysis Interdisciplinary Capability Knowledge Verification and Pattern validation Interface Management Recognition Written Communication Interpersonal Skills Planning Skills

1960s 1970s 1990s 2000s

Figure 11. Summary of Traits by Year of Publication (Literature Review)

When combined, the three data sets differentiate the skills expected from various levels of experience. Figure 12 illustrates the traits consistent with expectations from online classified advertisements. The experimental fuzzy logic scale was adapted from Moore’s (2000) definitions of Fat-Short and Thin-Tall Systems to describe the different types of Systems Engineers. Originally, the term was used by Mead and Conway (1979) to describe designers. Fat-Short is a person with a specialized set of technical skills who cannot easily integrate concepts from multiple disciplines. Thin-Tall was a person with a broad technical skill who can easily integrate concepts from multiple disciplines.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 14 of 18

Fat Thin

Short Tall

Total Systems Consideration / Production

Evaluation of Technical Plans / Evaluation of Systems

Engineering Analysis

Cost Management Product Development Process / Coaching Skills Matrix relationship understanding

Coordinating Skills Mentoring management staff

Quality Management

Functional Definition Modeling / Interface Management

IT Skills / Software Experience

Technical Skills / Attention to Details Documentation / System Architecture

Subject Matter Expert in an Engineering / Technical discipline / Trade Studies / Human Systems Integration

Decision Making Skills / Policy Making / Certification

Manage Teams / Organization Skills / Standards Familiarization

Contract Knowledge / Problem Solving Skills / System Design / Adaptable / Computer skills with SE Tools

Systems Thinking / Testing Skills

Analytical Skills / Handle Product Design Specifications / Leader / Multi Tasking

Verification and validation / Complex Systems Handling / Interdisciplinary Knowledge / Project Management Skills / Specific Domain Knowledge

Customer Management / Focus

Integration (Function, Systems, Technical) / Systems Engineering Process

Interpersonal Skills / Requirements Analysis / Risk Analysis

Oral / Written Communication Skills / Team Player

1 3 5 7 10 Low Mid-Low Mid Mid-High High 1 year 2 - 3 years5-7 years 10 years >15 years

Figure 12. Fuzzy Logic of Traits to Years of Experience (Classified Ads)

References

Handbook of qualitative research1994. eds. Norman K. Denzin, Yvonna S. Lincoln. California, USA: Sage Publications Inc. Andress, Frank J. 1954. The learning curve as a production tool. Harvard Business Review 32, (1) (01//Jan/Feb54): 87-97. Arnold, E. P. 2006. "Defining, Finding, and Hiring REAL Systems Engineers", proceedings of 16th Annual Symposium of the International Council on Systems Engineering, INCOSE, Orlando FL. Augustine, Norman R. October 2000. The engineering of systems engineers. IEEE Aerospace & Electronic Systems Magazine (Jubilee Issue). Baird, Jack A. 1971. How can systems engineers do systems engineering?. Aerospace and electronic systems, IEEE transactions on. Vol. AES-7. Bandemer, Hans, and Siegfried Gottwald. 1995. Fuzzy sets, fuzzy logic, fuzzy methods with applications. New York: Chichester ; J. Wiley. Barber, D. E. 1998. An overview of the systems engineering capability model EIA/IS 731. Digital avionics systems conference, 1998. proceedings., 17th DASC. AIAA/IEEE/SAE. Vol. 1.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 15 of 18

Bar-Yam, Y. 2003. When systems engineering fails-toward complex systems engineering. Systems, man and cybernetics, 2003. IEEE international conference on. Vol. 2. Boarder, J. C. 1995. The system engineering process. Engineering management conference, 1995. 'global engineering management: Emerging trends in the Asia Pacific'., proceedings of 1995 IEEE annual international. Booton, Richard C., and Simon Ramo. 1984. The development of systems engineering. Aerospace and electronic systems, IEEE transactions on. Vol. AES-20. Brill, James H. 1998. Systems engineering - A retrospective view. Systems Engineering 1, (4): 258-66. Chen, Pin, and Jennie Clothier. 2003. Advancing systems engineering for systems of systems challenges. Systems Engineering 6, (3). Chiang, H. Ren. 2003. Expanding the role of systems engineers in the age of escalating complexity. Systems Engineering 6, (2): 116. Covey, Stephen R. 15 September 1990. The seven habits of highly effective people. 1st edition ed.Free Press. Cosgrove, Peter. 2006. Salary Survey Ireland, Hudson. Dahmann, J. S., and K. J. Baldwin. 2008. Understanding the current state of US defense systems of systems and the implications for systems engineering. Systems conference, 2008 2nd annual IEEE. Davidz, Heidi L. 2005. Accelerating the development of senior systems engineers - PhD dissertation (MIT). de Souza, Roy. 2008. Maturity Curve of Systems Engineering, Thesis, Naval Postgraduate School, Monterey. Deming, W. Edwards. 1944. On errors in surveys. American Sociological Review 9, (4) (08): 359-69. Denzin, Norman K. 1989. The research act: A theoretical introduction to sociological methods. 3rd Edition ed. New Jersey: Prentice-Hall. ———. 1978. The research act: A theoretical introduction to sociological methods. 2nd Edition ed.McGraw-Hill. Emes, Michael, Alan Smith, and Douglas Cowper. 2005. Confronting an identity crisis: How to brand systems engineering. Systems Engineering 8, (2): 164-86. Feldman, Ronen, and James Sanger. December 11, 2006. The text mining handbook: Advanced approaches in analyzing unstructured dataCambridge University Press. FitzGerald, Tara. 2005. Learning by doing. Business Mexico 15, (7) (07): 32-. Frank, Moti. 2000. Engineering systems thinking & systems thinking. Systems Engineering 3, (3). Frank, Moti, Tony Di Carlo, and Keith Frampton. 2007. Characteristics of successful systems engineers, systems architects and IT architects.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 16 of 18

Frank, Moti, and David Elata. 2005. Developing the capacity for engineering systems thinking CEST of freshman engineering students. Systems Engineering 8, (2): 187-95. Frank, Moti, Ofer Zwikael, and Michael Boasson. 2007. Jobs requiring a capacity for engineering systems thinking (CEST): Selection using an interest inventory. Project Management Journal 38, (3) (09): 36-44. Garfinkel, Harold. 1984. Studies in Ethonomethodology. T.J. Press. Ltd. Padstow Gelbwaks, N. L. 1967. AFSCM 375-5 as a methodology for system engineering. Systems Science and Cybernetics, IEEE Transactions on 3, (1): 6-10. Goncalves, D. 2008. Developing systems engineers. Management of engineering & technology, 2008. PICMET 2008. portland international conference on. Hall, A.D. 1962. A methodology for systems engineering. Princeton, N.J.: D. Van Nostrand Co. Hart, Christopher. 1998. Doing a literature review: Releasing the social science imagination. 1 edition (March 1, 1999) ed.Sage Publications Ltd. Hearst, Marti. What is text mining? 2003 [cited 6 August 2008]. Available from http://people.ischool.berkeley.edu/~hearst/text-mining.html Janesick, Valerie J. 1994. The dance of qualitative research design. 209-219 Sage Publications. Jansma, P. A. “Trisha”, and Ross M. Jones. 2006. Advancing the practice of systems engineering at JPL. IEEEAC Version 6, paper #1031. Kaplan, David (Editor). 2004. The sage handbook of quantitative methodology for the social sciences. Thousand Oaks, Ca: Sage, c2004.: Sage Publications, Inc. Kasser, Joe. 1995. Applying total quality management to systems engineering. Norwood, MA, USA: Artech House. ———. Eight deadly defects in systems engineering and how to fix them. Paper presented at Proceedings of the 17th International Symposium of the INCOSE, San Diego, CA. ———. The hitchins-kasser-massie (HKM) framework for systems engineering. San Diego, CA. Kasser, Joe, and Angus Massie. 20 Jul 2001. A framework for a systems engineering body of knowledge. Paper presented at 11th International Symposium of the INCOSE, Melbourne, Aust. Kasser, Joe, and K. Palmer. Reducing and managing complexity by changing the boundaries of the system. Paper presented at The Conference on Systems Engineering Research, Hoboken NJ,. Kossiakoff, Alexander, and William N. Sweet. 2003. Systems engineering - principles and practice. Hoboken, New Jersey: John Wiley and Sons, Inc. Langford, Gary O., Raymond Franck, Tom Huynh, and Ira Lewis. 2008. Gap analysis: Rethinking the conceptual foundations. Monterey, California, Naval Postgraduate School. Lewkowicz, P. E. 1988. Effective systems engineering for very large systems: An overview. Aerospace applications conference, 1988. Digest, 1988 IEEE. Lodge, Milton. 1981. Magnitude scaling, quantitative measurement of opinions. Beverly Hills: Sage Publications.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 17 of 18

Martin, James N. 1996. Systems engineering guidebook : A process for developing systems and products. Boca Raton, Fl: CRC Press. Marx, Gary T. 1997. Of methods and manners for aspiring sociologists: 37 moral imperatives. American Sociologist 28, (1) (Spring97): 102-25. Mason, Jennifer. 2006. Qualitative researching. 2nd edition ed.Sage Publications Ltd. Mead, Carver, and Lynn Conway. 1979. Introduction to VLSI systems. New York, NY: Addison-Wesley Publishing Company. Mintz, R. B. 1994. Systems engineering fundamentals. Northcon/94 conference record. Moore, Robert C. 2000. Characteristics of a successful space system engineer. Aerospace and Electronic Systems Magazine, IEEE 15, (3): 21-7. Naval Center for Cost Analysis (NCCA). Naval center for cost analysis (NCCA). 2008 [cited 01 November 2008]. Available from http://www.ncca.navy.mil/services/inflation.cfm Pedrycz, Witold, and Fernando Gomide. 2007. Fuzzy systems engineering: Toward human- centric computing. Hoboken, New Jersey and simultaneously in Canada: John Wiley & Sons Inc. Raccoon, L. B. S. 1996. A learning curve primer for software engineers. ACM SIGSOFT Software Engineering Notes 21, (1 (January)) 77-86. Rittel, H. W., and M. M. Webber. 1973 Number. Dilemmas in a general theory of planning. Policy Sciences Vol 4, 155-69. Sheard, Sarah A. 1996. Twelve systems engineering roles. Proceedings of the INCOSE Sixth Annual International Symposium (Boston, Massachusetts, USA). Silverman, David. 2005. Doing qualitative research: A practical handbook. Second Edition ed. London: Sage Publications Ltd. Towill, Denis R. 1990. Forecasting learning curves. International Journal of Forecasting 6, (1) (03): 25-38. ———. 1985. Management systems applications of learning curves and progress functions. Engineering Costs & Production Economics 9, (4): 369-83. Vidale, R. F. 1970. University programs in systems engineering. Systems science and cybernetics, IEEE transactions on. Vol. 6. Williams, Christine, and Derro, Mary-Ellen. NASA systems engineering behavior study, Oct 2008 [cited 01 December 2008]. Available http://www.nasa.gov/news/reports/NASA_SE_Behavior_Study.html Zadeh, L. A. 1973. Outline of a new approach to the analysis of complex systems and decision processes. IEEE Trans. Systems, Man and Cybernetics 3, 28. ———. 1965. Fuzzy sets. Information and Control 8, (3) 338. Biographies Roy de Souza graduated with a Masters of Science in Systems Engineering from the Naval Postgraduate School, Monterey, California. He received his bachelors in Mechanical

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 18 of 18

Engineering from the National University of Singapore. He is currently a Major in the Singapore Armed Forces. INCOSE member.

Gary Langford is a lecturer in the Systems Engineering Department at the Naval Postgraduate School in Monterey, California. His research interests include the theory of systems engineering and its application to commercial and military competitiveness. Mr. Langford founded and ran five corporations – one NASDAQ listed. He was a NASA Ames Fellow. He has an A.B. in astronomy from UC Berkeley, and an M.S. in physics, Cal State Hayward. INCOSE member.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 21

Reengineering Systems Engineering

Joseph Kasser Derek Hitchins Thomas V. Huynh Visiting Associate Professor ASTEM Associate Professor Temasek Defence Systems Institute Consultant Department of Systems National University of Singapore Systems Engineering Block E1, #05-05, 1 Engineering Architect Naval Postgraduate School Drive 2, Singapore 117576 Monterey, CA, 94393 [email protected] [email protected]

Copyright © 2009 by the authors. Published and used by APCOSE with permission. Abstract. This paper documents (1) the early phases of using systems engineering to develop a conceptual system – the system being developed is a systems engineering body of knowledge (SEBoK) and (2) the findings and opportunities generated in those early phases. The approach was based on identifying activities specific to systems engineering, as opposed to the broad raft of activities that systems engineers might undertake, according to their role. An activity-based definition of systems engineering vs. non-systems engineering role-based definition was developed. The second part of the paper identifies five types of systems engineers, discusses the evolution of systems engineering in terms of those five types, and hypothesizes that a major cause of the failure of systems engineering is the allocation of inappropriate types of systems engineers to early lifecycle phase systems engineering activities. The paper concludes with some insights and recommendations for further study. Evolution of the role of systems engineering Descriptions of systems engineering currently comprise different interpretations of the activities known as systems engineering and the broad raft of activities that systems engineers might undertake according to their role in the workplace. This quagmire has developed because different users of the term ‘systems engineering’ for almost 50 years have chosen or perceived different meanings. For example, one comment from 1960 was “Despite the difficulties of finding a universally accepted definition of systems engineering1, it is fair to say that the systems engineer is the man who is generally responsible for the over-all planning, design, testing, and production of today’s automatic and semi-automatic systems” (Chapanis, 1960) page 357). (Jenkins, 1969) page 164) expanded that comment into the following 12 roles of the systems engineer: 1. He tries to distinguish the wood from the trees – what’s it all about? 2. He stimulates discussion about objectives – obtains agreement about objectives. 3. He communicates the finally agreed objectives to all concerned so that their co-operation can be relied upon. 4. He always takes an overall view of the project and sees that techniques are used sensibly. 5. By his overall approach, he ties together the various specializations needed for model building. 6. He decides carefully when an activity stops.

1 Fifty years later, nothing has changed in that respect. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 21

7. He asks for more work to be done in areas which are sensitive to cost. 8. He challenges the assumptions on which the optimization is based. 9. He sees that the project is planned to a schedule, that priorities are decided, tasks allocated, and above all that the project is finished on time. 10. He takes great pains to explain carefully what the systems project has achieved, and presents a well-argued and well-documented case for implementation. 11. He ensures that the users of the operational system are properly briefed and well trained. 12. He makes a thorough retrospective analysis of systems performance.

Seven of these roles of the systems engineer (activities performed by a person with the title systems engineer) overlap the role of the project manager (activities performed by a person with the title project manager). Research into the reason for the overlapping of the disciplines turned up information as to how the overlap originated in the form of the following statement. “Driven by cold war pressures to develop new military systems rapidly, operations research, systems engineering, and project management resulted from a growing recognition by scientists, engineers and managers that technological systems had grown too complex for traditional methods of management and development” (Johnson, 1997) Thus systems engineering, project management and operations research can be seen as three solutions to the problems posed by complex systems in the Cold War by three different communities of practice (Johnson, 1997) that have continued to evolve and overlap. Some of the evolution in systems engineering can be seen in the very little overlap between the 12 roles documented by (Jenkins, 1969) and the following 12 systems engineering roles documented by (Sheard, 1996): 1. Requirements Owner (RO) Role. Requirements Owner/requirements manager, allocator, and maintainer/specifications writer or owner/developer of functional architecture/developer of system and subsystem requirements from customer needs. 2. System Designer (SD) Role. System Designer/owner of “system” product/chief engineer/system architect/developer of design architecture/specialty engineer (some, such as human-computer interface designers)/“keepers of the holy vision” (Boehm, 1994). 3. System Analyst (SA) Role. System Analyst/performance modeler/keeper of technical budgets/system modeler and simulator/risk modeler/specialty engineer (some, such as electromagnetic compatibility analysts). 4. Validation and Verification (VV) Role. Validation and Verification engineer/test planner/owner of system test program/system selloff engineer. VV engineers plan and implement the system 5. Logistics and Operations (LO) Role. Logistics, Operations, maintenance, and disposal engineer/developer of users’ manuals and operator training materials. 6. Glue (G) Role. Owner of “Glue” among subsystems/system integrator/owner of internal interfaces/seeker of issues that fall “in the cracks”/risk identifier/“technical conscience of the program”. 7. Customer Interface (CI) Role. Customer Interface/customer advocate/customer surrogate/customer contact. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 21

8. Technical Manager (TM) Role. Technical Manager/planner, scheduler, and tracker of technical tasks/ owner of risk management plan/product manager/product engineer. 9. Information Manager (IM) Role. Information Manager (including configuration management, data management, and metrics). 10. Process Engineer (PE) Role. Process engineer/business process reengineer/business analyst/owner of the systems engineering process. 11. Coordinator (CO) Role. Coordinator of the disciplines/tiger team head/head of integrated product teams (IPTs)/system issue resolver. 12. “Classified Ads Systems Engineering” (CA) Role. This role was added to the first eleven in response to frustration encountered when scanning the classified ads, looking for the INCOSE-type of systems engineering jobs. Jenkins’ roles relate to conceiving and planning the solution system while almost 30 years later, few of Sheard’s roles address the original systems engineering approach to conceiving and planning the solution system. Sheard’s set of roles relate to interpersonal relationships between the practitioners of disparate skills and disciplines implementing the solution system. Furthermore, according to both Jenkins and Sheard the role of the systems engineer (the activities performed by a person with the title systems engineer) overlaps activities performed (the roles) by people from other professions2; see (Brekka, et al., 1994; Roe, 1995; DSMC, 1996; Kasser, 1996; Sheard, 1996; Johnson, 1997; Watts and Mar, 1997; Bottomly, et al., 1998; Kasser, 2002b) and Figure 1 for just a few examples of the different overlaps between systems engineering and project management. Note the Defense Systems Management College definition of systems engineering as “The management function which controls the total system development effort for the purpose of achieving an optimum balance of all system elements. It is a process which transforms an operational need into a description of system parameters and integrates those parameters to optimise the overall system effectiveness” (DSMC, 1996). Notice the use of the term “management function”! In addition, see (Emes, et al., 2005) for overlaps between systems engineering and other disciplines and (Hari, et al., 2004) for an example of

Figure 1 JAXA Project management and systems engineering (JAXA, 2007)

2 A different set, as seen across the years. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 21

the activities performed in new product design that overlap systems engineering. In addition, the activities performed by the systems engineer in one organisation are different to those performed by a systems engineer in another organization and so are the knowledge requirements for their activities. Consequently, defining a body of knowledge for systems engineering poses a major challenge. Defining a body of knowledge based on the role of a systems engineer will be difficult if not impossible because the role of the systems engineer has evolved over time so that it is different in practically every organisation. As such, the solution to the problem of defining a body of knowledge for systems engineering is to dissolve the problem by making a change in the paradigm. This approach, which redesigns the system containing the problem or changes the perspective from which the problem is viewed to produce an innovative solution is one of the four ways to tackle a problem suggested by (Ackoff, 1999) page 115). The paradigm change is made by making a distinction between a set of activities known as systems engineering and the role of the systems engineer which is the sum of the systems engineering and non-systems engineering activities systems engineers perform in the workplace (Kasser and Palmer, 2005). The focus on activities is a return to Hall’s definition of “systems engineering as a function not what a group does” (Hall, 1962) page 11) and means that the knowledge needed by systems engineers in their roles will be more than the activities to be defined as ‘systems engineering’ (see below), and that knowledge can be separated into ‘systems engineering’ and ‘everything else’. The ‘systems engineering’ knowledge will be placed in the systems engineering body of knowledge (SEBoK), and the subset of knowledge of ‘everything else’ that will be needed will be out of scope of the SEBoK but will be referenced appropriately. Separating out the systems engineering knowledge In the activity paradigm, various people in various disciplines at various times perform a set of activities from the time a problem is being defined, though the conceptualisation, design, construction and operation of the system that solves, resolves or dissolves the problem to the time that the system has been taken out of service and disposed. This set of activities may be partitioned into subsets in various ways such as by professional discipline (project/engineering management, systems engineering, engineering, new product design, etc.) and by time (the phases in the system lifecycle). Various systems engineers and non-systems engineers perform different subsets of systems engineering activities and different subsets of non- systems engineering activities. As discussed above, the mapping of the role of the systems engineer to activities is different in different organisations, hence the aforementioned difference in their descriptions when systems engineers get together and discuss their roles. Looking at the structure of organisations from the temporal perspective, in general the structure of organisations is still based on the work of F.W. Taylor who systems engineered his mining organisation and split the work into two streams of activities which have become known as ‘management’ and ‘labour’(Taylor, 1911). However, since that time, the structure of companies and the nature of work have changed. Organizational structures have become flatter, decision making has become decentralized, information is widely shared, workers form project teams, even across organizations, and work arrangements are flexible (Microsoft, 2008). Taylor’s split is no longer applicable. Consequently, we now propose to reengineer (in the sense of the word as used by (Hammer and Champy, 1993) Taylor’s split for organisations Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 21 developing systems by splitting work into two different streams, systems engineering and non-systems engineering and further partitioning the non-systems engineering streams as described below. The INCOSE Fellows definition of systems engineering was considered as a starting point for determining what went into the systems engineering stream. The definition is “Systems Engineering is an engineering discipline whose responsibility is creating and executing an interdisciplinary process to ensure that the customer and stakeholder's needs are satisfied in a high quality, trustworthy, cost efficient and schedule compliant manner throughout a system's entire life cycle” (INCOSE Fellows, 2009). However, if the words ‘engineering discipline’ are replaced by the words ‘project management’ many project managers would consider the definition to apply to project management. This definition may be understood as applying to both the role of the systems engineer and the role of the project manager since the roles overlap both in space and time as discussed above. As such, an alternative definition is needed. Further research to determine what went into the systems engineering stream showed that the approved Standards used in systems engineering do not seem to actually apply to systems engineering – they cover systems engineering management and the processes for engineering a system! Thus:  Mil-STD-499 covers systems engineering management (MIL-STD-499, 1969).  Mil-STD-499A covers engineering management (MIL-STD-499A, 1974) dropping the word ‘systems’ from the title.  The draft (MIL-STD-499B, 1993) and MIL-STD-499C (Pennell and Knight, 2005) Standards contain the words “systems engineering” in their titles but the Standards were never approved and these Standards also (as did 499 and 499A) generally ignore most of the problem identification, whole solution conceptualisation and solution implementation planning activities that take place in Phase A of Figure 1.  ANSI/EIA-632 covers processes for engineering a system (ANSI/EIA-632, 1999).  The IEEE 1220 Standard is for the application and management of the systems engineering process (IEEE 1220, 1998).  The ISO/IEC 15288 Standard lists processes performed by systems engineers (Arnold, 2002) and hence may be considered as being applicable to the role of the systems engineer rather than to the activities known as systems engineering. The phases in providing a whole complete solution to a problem can be considered as a set of activities performed by various people in various disciplines at various times. Some of those activities are systems engineering, and some are not systems engineering. The next approach was to develop a list of activities that could be described as systems engineering. Research found several sources of lists of activities including:  (Eisner, 1988) who lists a general set of 28 tasks and activities that were normally performed within the overall context of large-scale systems engineering. He calls the range of activities ‘specialty skills’ because some people spend their careers working in these specialties. Thus according to Eisner [the role of3] systems engineering overlaps at least 28 engineering specialties.

3 Author’s interpretation. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 21

 (Hyer, 1997) provides a list of nine activities for systems integration but which do not necessarily take place during the systems integration phase.  (Eisner, 1997) page 156) expands (Eisner, 1988) and discusses 30 tasks that form the central core of systems engineering. The whole area of systems engineering management is covered in just one of the tasks. Eisner states that “not only must a Chief Systems Engineer understand all 30 tasks; he or she must also understand the relationships between them, which is an enormously challenging undertaking that requires both a broad and deep commitment to this discipline as well as the supporting knowledge base”. Should the research continue in this direction, the resulting list would be long, subjective and open to never ending discussion. Looking outside the box, lessons learned from psychology indicate that long lists are not the way to proceed. At one point of time in the development of theories of motivation, Henry A. Murray identified separate kinds of behaviour and developed an exhaustive list of psychogenic or social needs (Murray, 1938). However, the list is so long that there is almost a separate need for each kind of behaviour that people demonstrate (Hall and Lindzey, 1957). While Murray’s list of 39 kinds of behaviours has been very influential in the field of psychology, it has not been applied directly to the study of motivation in organizations because the length of the list makes it impractical to use. On the other hand, Maslow's hierarchical classification of needs (Maslow, 1966, 1968, 1970) has been by far the most widely used classification system in the study of motivation in organizations. Maslow differs from Murray in two important ways; his list is:  Arranged in a hierarchy -commonly drawn as a pyramid, and contains a set of hypotheses about the satisfaction of these needs.  Short -- Only five categories. The eventual approach chosen to determine what is and what is not a systems engineering activity was to dissolve the problem by developing a criterion for what constitutes an activity to be defined as systems engineering rather than trying to resolve the problem by a developing a list of activities. The following criterion was used to determine if an activity does or does not belong in the set of activities to be known as systems engineering:  If the activity deals with parts and their interactions as a whole, then it is an activity within the set of activities to be known as systems engineering.  If the activity deals with a part in isolation, then the activity is not an activity within the set of activities to be known as systems engineering but is part of ‘something else’, e.g., engineering management, software engineering, etc. The activities of systems engineering have focused on both analysis and systems thinking. Analysis which has three steps (Ackoff, 1991) can be performed as ‘reductionism’ or ‘decomposition’ – reducing the parts to ever decreasing components in isolation, but should be performed by the systems engineer as ‘elaboration’ (Hitchins, 2003) pages 93-95) – which examines the parts in increasing detail without losing track of the part’s relationship to the overall system. Systems thinking, on the other hand also has three steps (Ackoff, 1991) but they are slightly different. Comparing analysis and systems thinking in the manner shown in Table 1, one can see that the focus of analysis is to look inwards while the focus of systems thinking is to look outwards. Both analysis (in the form of ‘elaboration’) and systems thinking have their place in the activities performed in developing an understanding of a Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 21

Table 1 Analysis and Systems Thinking Analysis (Machine Age) Systems Thinking (Systems Age) 1. Take apart the thing to be 1. A thing to be understood is understood conceptualized as a part of one or more larger wholes, not as a whole to be taken apart; 2. Try to understand how these parts 2. An understanding of the larger system worked is sought; 3. Assemble an understanding of the 3. The system to be understood is parts into an understanding of the explained in terms of its role or whole. function in the containing system.

system (Hitchins, 1992) page 14) and are but two of the systems thinking perspectives (Kasser and Mackley, 2008). Since the activities forming the ‘something else’s’ are part of the context of systems engineering and are often performed by systems engineers, it is recognised that systems engineers need the knowledge to perform or understand many of the activities defined as ‘something else’ but that knowledge per se is out of the scope of the SEBoK and will be identified accordingly. The ‘something else’ activities were further partitioned into the following sets of non-systems engineering activities:  Engineering.  Management.  Other. The proposed activity paradigm definitions of the systems engineering sets of activities and the non-systems engineering sets of activities are as follows:  Systems engineering is the set of activities involved with dealing with parts and their interactions as a whole.  Engineering is the set of activities dealing with a part in isolation. If the part is not a technological product, for example if the part is such as a human element, then use of language is such that the activity is not called engineering but something else, such as training or exercising.  Management is the set of activities known as planning organising, directing, staffing and controlling activities for and in the production of the part in isolation.  Other is the remaining set of activities not included in the previous definitions. Combining these definitions it can be seen that in the activity paradigm:  Systems engineering management is the set of activities known as planning organising, directing, staffing and controlling systems engineering activities in isolation from the other sets of management activities.  Engineering management is the set of activities known as planning organising, directing, staffing and controlling engineering activities in isolation from the other sets of management activities. Lastly for the sake of completing the set of definitions, a task is an activity performed within a specific period of time and a project consists of a temporary endeavor [set of tasks] undertaken to create a unique product, service or result (PMI, 2004). It follows that: Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 21

 Project management is the set of activities known as planning organising, directing, staffing and controlling a temporary set of tasks undertaken to create a unique product, service or result, in isolation from the other projects. The next phase in determining the SEBoK will be to identify the activities performed in each phase of the system lifecycle developing a concept of operations of the work being performed and then using the simple activity-based criterion to determine which of the activities are and which are not systems engineering. Each activity will be defined in such a manner as to terminate with the production of a tangible product or products which is/are transferred to the start of the subsequent activity in accordance with (Kasser, 1997). The activities have been grouped by the phases in the first and second systems engineering processes4 in the system lifecycle for a system that is developed from conception to disposal5 using the Hitchins-Kasser-Massie Framework (HKMF) for understanding systems engineering (Kasser, 2007) shown in Figure 2. Each area of the HKMF can potentially contain all sets of (systems engineering and non-systems engineering) activities – some more than others. Figure 1 also provides an indication of the relative ratios between the sets of activities known as systems engineering and the sets of activities known as project management over the system life cycle. The HKMF has also identified one reason for debates in the meaning of terminology used by systems engineers. Words such as ‘capability’ and ‘system design’ have different meanings in different areas of the HKMF. The confusion in the use of the term ‘operations concept’ and ‘concept of operations’ can be similarly be clarified when one realizes that the terms refer to products produced in different columns of the

Figure 2 The HKM Framework for understanding systems engineering

4 The first systems engineering process deals with identifying the real problem and a number of alternative conceptual solutions followed by the choice of an optimal conceptual solution to the whole problem, The second systems engineering process follows the first and deals with the creation, operation and disposal of an optimal physical implementation of the conceptual solution to the problem generated by the first systems engineering process. 5 Other lifecycles do exist. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 21

HKMF. In addition the vocabulary for describing concepts in Layer 2 for single system development in isolation is different to the vocabulary used in Layer 3 to express the same concepts in business processing reengineering. The vertical dimension of the HKMF contains the five-layers of systems engineering (Hitchins, 2000). The horizontal dimension of the framework is organized as sequential phases in providing a whole complete solution to a problem as an overall, end-to-end process which consists of conceiving a whole solution to solve a problem and making that whole “come to life” for the development of a single system in isolation. The phases have been stated in various ways in various Standards, conference papers and books, but in the HKMF they are defined in generic terms as: A. Identifying the need. This is the phase where the bulk of the set of activities known as systems engineering is performed. Yet in the Type II systems engineering educational paradigm it tends to be glossed over (see below for an explanation of the five types of systems engineers and why this phase is glossed over). Phase A is based on (Hall, 1962), (Gelbwaks, 1967), (Hitchins, 1992) and the summary in (Brill, 1998) and contains the first ‘systems engineering’ process addressing the conceptual solution. Phase A comprises the following sub-phases: 1. This sub-phase contains the set of activities that explore/scope the problem, leading directly to Phase A.2. The activities performed in this phase produce a definitive statement of the problem-in-context. 2. This sub-phase contains the set of activities that conceive the whole solution system (which 'emerges' from/"complements" the problem) and produces the concept of operations (CONOPS) that describes how the solution system will operate in its future environment. 3. This sub-phase contains the set of activities that design the whole solution system, identify the environment, other interacting systems, the subsystems, parts, interactions, functional architecture, physical architecture, etc., etc., - but still all of the whole. B. Requirements analysis. This phase is the first phase of the second ‘systems engineering’ process addressing the physical solution and its implementation and contains the set of activities that specify the solution system as a full set of specifications for the whole and for the parts and their infrastructure, including the environment/Weltanschauung or paradigm that justifies them. If the specifications are in the form of text mode requirements, the output of this phase tends to be at the ‘A’ specification level (MIL-STD-490A, 1985). Unfortunately, many systems engineers have been educated to consider this phase as the first phase of a single systems engineering process. For example, (1) according to (Martin, 1997) page 95), (Eisner, 1997) page 9), (Wasson, 2006) page 60) and (DOD 5000.2-R, 2002), pages 83-84) requirements are one of the inputs to the ‘systems engineering process’; and (2) in one postgraduate class at University of Maryland University College the instructor stated that systems engineering began for him when he received a requirements specification (Todaro, 1988). While (DOD 5000.2-R, 2002) pages 73-74 ) does call out the ‘analysis of possible alternatives’ subset of activities in Phase A2 of the HKMF, those activities are called out as part of the separate seemingly independent Cost as an Independent Variable (CAIV) process which (1) is a way of complicating just a part of the concept of designing budget tolerant systems (Denzler and Kasser, 1995) using the Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 21

Cataract approach (Kasser, 2002a) and (2) takes place before the DOD 5000.2-R ‘systems engineering process’ begins. C. Design. This phase contains the set of activities that creates a more detailed design of the whole solution system through a combination of people, doctrine, parts, subsystems, interactions, etc., including configuration, architecture and implementation criteria. The output of this phase tends to be at the ‘B’ specification level (MIL-STD-490A, 1985). D. Construction. This phase contains the set of activities that create the individual parts, subsystems, interactions, etc. in isolation. Consequently the set of activities are mainly engineering, training, etc., not systems engineering. This situation is indicated in Figure 1 by the down slope in the line showing the amount of systems engineering at this phase. E. Unit Testing. This phase contains the set of activities that validate the performance of the individual parts, subsystems, interactions, etc. in isolation against their requirements. Consequently the set of activities are mainly engineering, not systems engineering. F. Integration and testing of the system. This phase contains the set of activities that (1) combines the parts, subsystems, interactions, etc., to constitute the solution system, and (2) establishes, under test conditions, the performance of the whole solution system, with optimum effectiveness, in its operational context. G. Operations, maintenance and upgrading of the system. This phase contains the set of systems engineering and non-systems engineering activities that actively provide a solution to the problem for which the whole system was created. This phase includes operating the system, support to maintain operations; improvements to the whole to enhance effectiveness, and to accommodate changes in the nature of the problem over time. These changes iterate phases A to F (call them Ga .. Gf), ideally without rendering the operating solution system materially inoperative for an unacceptable period of time. H. Disposal of the system. This phase contains the set of activities that dispose of the system. This phase is rendered necessary where either where the problem no longer exists, or the solution system is no longer capable of solving the problem effectively or economically. If the disposal method has not been predetermined, this phase may also iterate phases A to F (call them Ha .. Hf). This approach to determining the contents of the SEBoK is also domain independent but recognises that systems engineers do need domain knowledge (as well as systems thinking, communications and interpersonal skills). A serendipitous outcome of this approach which needs more research, would truly reengineer the work of (Taylor, 1911). For example,  The potential exists to redraw role boundaries to align with the activity boundaries and remove much of the role overlap and inefficiency in organisations.  A systems engineering approach can be used to determine the systems and non-systems engineering activities performed in any row and column of the HKMF based on the operations performed in that area of the framework. The activities can be grouped in various ways into specific roles (job positions) and the knowledge requirements for those roles can be developed. These Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 21

requirements would provide the knowledge component requirement for the person or persons to be assigned to perform the activities. The competency requirement for the person would be determined separately. The five types of systems engineers The human side of systems engineering is the systems engineers who perform the roles known as systems engineering. These roles perform the conceiving and creating the solution system systems engineering activities, the project management activities, engineering and other speciality engineering activities in various mixes depending on the phase in the system lifecycle and the organisation in which the systems engineer works. Optimal performance of each of the activities requires different characteristics in the systems engineer. Previous attempts to identify characteristics of systems engineers have been based on the traits attributable to systems engineers e.g. (Hall, 1962; Frank, 2006) and the INCOSE UK Systems Engineering Competencies Framework (Hudson, 2006) The list of desirable traits is increasing steadily. However, the lessons learned from psychology discussed above suggest lists are not the way to proceed and that an alternate approach be found. Hence, instead of using lists of traits, an alternative approach6 of characterising systems engineers into the following five types is proposed based on their ability to deal with problems and solutions.  Type I. This type is an “apprentice who can be told “how” to implement the solution and can then implement it.  Type II. This type is the most common type of systems engineer. Type II’s have the ability to use the systems engineering process to figure out how to implement a physical solution once told what conceptual solution to implement.  Type III. Once given a statement of the problem, this type has the necessary know-how to conceptualize the solution and to plan the implementation of the solution.  Type IV. This type has the ability to examine the situation and define the problem (Wymore, 1993) page 2).  Type V. This type combines the abilities of the Types III and IV, namely has the ability to examine the situation, define the problem, conceptualise the solution and Table 2 Failure data from GAO Report 06-368, 2006

6 Based on years of observations by the authors. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 12 of 21

plan the implementation of the physical solution. Types I to III are levels through which a person grows with education and experience. The debate on ‘nature’ or ‘nurture’ comes into play at Levels IV and V. However, irrespective of the debate, it is important to identify people with the potential to become Type IV’s and V’s as early as possible in their careers and then to provide them with fast track training to enable their organization to obtain the best use of their capabilities in the future. The new approach to characterizing systems engineers provides a hypothesis for a reason for the failure of systems engineering in the early stages of large projects (Hiremath, 2008) and other examples of poor systems engineering implementation (GAO, 2006). For example, the cost and schedule overruns in the Joint Strike Fighter (JSF) development project shown in Table 2 were predicted in (Kasser, 2001) and hence probably preventable. Had Type V systems engineers been working on the phases of the JSF project in column A of the HKMF, the factors identified as potential causes of cost and schedule overruns leading to the prediction in (Kasser, 2001) would have probably been identified as risks. Appropriate risk management techniques would then have been recommended and if these risk management techniques had been implemented7, the ensuring cost and schedule overruns would have been reduced. Research seems to show that the early systems engineers of the 1950’s and 1960’s tended to focus on identifying the problem (Wymore, 1993) and finding an optimal solution (Goode and Machol, 1959; Hall, 1962). These early systems engineers were of Type III, IV, and V, while the systems engineers who came later tended to focus on processes (Type II)’s. Back in the “good old days” of systems engineering Type III, IV and V systems engineers solved/resolved/dissolved the problem in the first ‘systems engineering’ process addressing the conceptual solution, then initiated the implementation of the solution, and moved on to the next contract, leaving the Type II’s to continue assisting the development of the solution system in the second systems engineering process. There then came a time when there was a lack of new projects and so many of the Type III, IV and V’s were laid off and lost to the discipline. When the need for systems engineers picked up again, in general only the Type II systems engineers were left and they took over systems engineering. They had seen a successful process for developing systems and so their focus was on the second systems engineering process. They wrote the standards used in systems engineering (MIL-STD-499, 1969; MIL-STD-499A, 1974; EIA 632, 1994; IEEE 1220, 1998) for other Type II systems engineers to follow. These Standards in turn became the foundation for educating systems engineers. The 499, 499A, 632, 1220, and 15288 Standards cover the systems engineering process and engineering management because there is actually very little systems engineering (the activity not the role) in the subsystem design, construction, and unit testing phases (HKMF Columns C, D and E) of the systems lifecycle for a single system in isolation. Activities pertaining to subsystems and units in isolation are engineering of systems not systems engineering activities according to the criterion defined above. The mantra became ‘follow the process and all will be well’. The term GIGO - garbage in, garbage out, was acknowledged but ignored. In this paradigm:

7 A big “if” since political considerations in the Type II process paradigm would probably have precluded the risk mitigation activities. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 13 of 21

Table 3 Focus of Standards – Chronological Order MIL-STD ANSI IEE CMMI ISO 499C EIA 632 1220 15288 Conceptualizing problem No No No No No and alternative solutions Mission Purpose/Definition No No Yes Yes Yes Requirements engineering Yes Yes Yes Yes Yes System architecting Yes Yes Yes Yes Yes System implementation No Yes No Yes Yes Technical analysis Yes Yes Yes Yes Yes Verification & validation Yes Yes Yes Yes Yes

 While the subset of the systems engineering profession focuses on processes (Type II systems engineers), the literature on “excellence” focuses on people (Type V systems engineers) e.g. (Peters and Waterman, 1982; Peters and Austin, 1985; Rodgers, et al., 1993).).  The focus is on process and not on providing an understanding of the context and the ability to tailor the process (as was called out in (MIL-STD-499, 1969). This is seen in systems engineering courses where the students are taught about the process but not about the context.  Processes seen to work in one culture or organization have been copied verbatim by other organizations, with dismal results. Examples can be found in the lessons learned in (O’Toole, 2004) and (Angel and Froelich, 2008)’s reasons for a claimed Six Sigma initiative 60% failure rate.  The systems engineering process has a high degree of correlation to the problem solving process because that was the process documented in the Standards. See (GDRC, 2009) and (OVAE, 2005) for typical examples of the problem solving process.  The Standards commonly used/taught in systems engineering (MIL-STD-499, 1969; MIL-STD-499A, 1974) and (DOD 5000.2-R, 2002) pages 83-84) ignore most of the activities allocated to Phase A in Figure 1 and Phase A of the HKMF resulting in the critical first systems engineering process addressing the conceptual solution being out of mainstream Type II systems engineering. Table 3 contains data extracted from Table 5 in (Honour and Valerdi, 2006) and rearranged in chronological order8 showing the lack of coverage of the mission purpose/definition activities in MIL-STD 499 and ANSI EIA 632. The top row in Table 3 has been added in this paper to show that MIL-STD 499 and ANSI EIA 632 do not cover the conceptual activities in the first systems engineering process and while the Systems Engineering Capability Maturity Model (CMM), the draft MIL-STD-499C Standard and ISO 15288 do address the mission/purpose definition activities to some extent they also do not cover the conceptual activities in the first systems engineering process. This situation (addressing mission/purpose definition activities to some extent while failing to cover the first systems engineering process) also appeared in a survey of current systems engineering processes in (Bruno and Mar, 1997) and in (Fisher, 1996)’s list of the engineering and systems engineering activities assigned to the systems

8 Based on the issue date of MIL-STD-499, not the draft MIL-STD-499C since the contents of MIL- STD 499A and MIL-STD-499B don’t differ from MIL-STD 499C in this respect. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 14 of 21

Table 4 Types of Knowledge (Woolfolk, 1998) Declarative knowledge Knowledge that can be declared in some manner. It is “knowing that” something is the case. Describing a process is declarative knowledge. Procedural knowledge Knowing how to do something. It must be demonstrated; performing the process demonstrates procedural knowledge. Conditional knowledge Knowing when and why to apply the declarative and procedural knowledge.

engineering organization/team based on the (MIL-STD-499B, 1993)/(EIA 632, 1994) Standards. A benchmark of systems engineering postgraduate degree syllabi A benchmark of systems engineering postgraduate degree syllabi seems to indicate that:  Much of systems engineering is now taught as declarative and procedural knowledge (See Table 4) (Woolfolk, 1998) describing the second systems engineering process. To be fair, this is not unique to systems engineering (Microsoft, 2008). For example, Peter Drucker wrote “Throughout management science--in the literature as well as in the work in progress--the emphasis is on techniques rather than principles, on mechanics rather than decisions, on tools rather than on results, and, above all, on efficiency of the part rather than on performance of the whole."(Drucker, 1973) page 509.) Today’s academic institutions seem to be producing Type II systems engineers and managers (engineer leaders); but they should be producing or at least identifying personnel with Type V characteristics by teaching conditional knowledge.  Some academic institutions teaching systems engineering are leaving out the critical first systems engineering process of HKMF Column A. For example, a proposed reference curriculum for systems engineering (Jain and Verma, 2007) begins in Column B of the HKMF. This reference curriculum complies with (Martin, 1997) page 95), (Eisner, 1997) page 9), (Wasson, 2006) page 60) and (DOD 5000.2-R, 2002), pages 83-84) which consider requirements as one input to the systems engineering process as mentioned above. This failure to teach the critical first systems engineering process has resulted in (1) at least one generation of “systems engineers” who are unfamiliar with the critical activities in Column A of the HKMF and (2) the terms CONOPS and ‘operations concept’ being used interchangeably by some systems engineers who do not have an appropriate frame of reference to understand the difference between the two documents when old timers try to explain it to them. Hypothesis for a reason for the failure of systems engineering Based on a combination of the five types of systems engineers and the history of systems engineering paraphrased in terms of those five types, the hypothesis is that a Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 15 of 21

current cause of failures in systems engineering is the assignment of Type II systems engineers or higher types trained in a Type II process thinking paradigm to tasks that need the problem/solution characteristics of the Type III, IV and V systems engineers. The associated prediction to test the hypothesis is that the cost and schedule overruns and other failures will continue in spite of all the funding being allocated to systems engineering education if the education of engineer leaders remains in the Type II paradigm and starts with the activities in column B of the HKMF. Type II systems engineers are and should be doing the engineering of systems (following the process designed by the Type V systems engineers). Type V systems engineers should be doing systems engineering in Columns A, B, F and G of the HKMF. Recommendations The following recommendations are made to improve systems engineering based on the research so far: 1. Continue with the development of the SEBoK creating a concept of operations for the product producing activities in each rectangle of the HKMF using the simple activity-based criterion to determine which of the activities are and which are not systems engineering and then defining the knowledge requirements for the activities known as systems engineering. 2. Investigate the potential of redrawing role boundaries to align with the activity boundaries and remove much of the role overlap and inefficiency in organisations. This approach, which needs more research, would truly reengineer the work of F. W. Taylor (Taylor, 1911). 3. Once the activities performed by systems engineers in each area of the HKMF have been identified, an appropriate level of competence for the activity should be made and optimal systems engineering teams could then be designed. 4. Work with psychologists to identify characteristics of the five types of engineer leaders so that the Type V’s may be identified early in their career and put through fast track training to increase their value to their organizations. 5. Modify the curriculum for teaching systems engineering to include activities enabling the early identification of potential Type V’s. 6. Modify the curriculum for teaching systems engineering to include the system engineering activities performed in Column A of the HKMF. 7. Develop a good set of educational materials for use with the modified curriculum. 8. Identify the activities performed in the non-systems engineering streams in each column of the HKMF. Then determine the knowledge and type of engineer leader needed to make optimal decisions and quantify the risks associated with decision making with specific levels of imperfect knowledge and using the wrong type of engineer leader. This model should inform customers concerning the prediction of the probability of future project failure at any point in any column of the HKMF by comparing the situation in a real project with the data in the model. Summary This paper documented the early phases of using systems engineering to develop a SEBoK and some of the findings. The first part of the paper discussed the nature of the problem and dissolved the problem by applying an out-of-the-box approach. The second part of the paper identified five types of systems engineers, discussed the evolution of systems engineering in terms of those five types, hypothesised that a major cause of the failure of systems engineering is the allocation of inappropriate Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 16 of 21 types of systems engineers to systems engineering activities and identifed a critcal gap in systems engineering education. The paper concluded with some insights from the out-of-the-box approach and recommendations for further study. Conclusion The out-of the-box approach to developing the SEBoK seems to be achievable and has produced some interesting insights. Biographies Joseph Kasser has been a practicing systems engineer for nearly 40 years and an academic for about 10 years. He is an INCOSE Fellow, the author of “A Framework for Understanding Systems Engineering” and "Applying Total Quality Management to Systems Engineering" and many INCOSE symposia papers. He is a recipient of NASA’s Manned Space Flight Awareness Award (Silver Snoopy) for quality and technical excellence for performing and directing systems engineering and other awards. He holds a Doctor of Science in Engineering Management from The George Washington University, is a Certified Manager and holds a Certified Membership of the Association for Learning Technology. He has also served as the initial president of INCOSE Australia and Region VI Representative to the INCOSE Member Board. He gave up his positions as a Deputy Director and DSTO Associate Research Professor at the Systems Engineering and Evaluation Centre at the University of South Australia in early 2007 to move to the UK to develop the world’s first immersion course in systems engineering as a Leverhulme Visiting Professor at Cranfield University. He is currently a principal at The Right Requirement Ltd. in the UK and a Visiting Associate Professor at the National University of Singapore. Derek Hitchins retired from full time academic work in 1994 on medical grounds, and is now a part-time consultant, teacher, visiting professor and international lecturer. Formerly, he held the British Aerospace Chairs in Systems Science and in Command and Control, Cranfield University at RMCS Shrivenham. Prior to that, he held the Chair in Engineering Management at City University, London. Derek started as a Cranwell apprentice and retired as a wing commander from the Royal Air Force after 22 years, to join industry. His first industry appointments were as the System Design Manager of the Tornado F3 Avionics, Technical Co-ordinator for UKAIR CCIS, and UK Technical Director for the NATO Air Command and Control System (ACCS) project in Brussels. He subsequently held posts in two leading systems engineering companies as Marketing Director, Business Development Director and Technical Director before becoming an academic in 1988. His current research is into system thinking, system requirements, social psychology & anthropology, Egyptology, command & control, system design and world-class systems engineering. He has published three systems engineering books: "Putting Systems to Work", John Wiley & Sons, in 1992; "Advanced Systems Thinking, Engineering and Management," Artech House, 2003; and, “Systems Engineering: A 21st Century Systems Methodology,” John Wiley & Sons in 2007/2008. He inaugurated the IEE’s PG M5 — Systems Engineering. He also started the UK Chapter of INCOSE and was its inaugural president. He is an INCOSE Fellow, an INCOSE "Pioneer" and a Charter Member of the Omega Alpha Association. Thomas V. Huynh is an associate professor of systems engineering at the Naval Postgraduate School in Monterey, CA. His research interests include complex systems Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 17 of 21

and complexity theory, system-of-systems engineering methodology, and systems architecting. Prior to joining the Naval Postgraduate School, Dr. Huynh was a Fellow at the Lockheed Martin Advanced Technology Center, Palo Alto, CA, where he engaged in research in computer network performance, computer timing control, bandwidth allocation, heuristic algorithms, nonlinear estimation, perturbation theory, differential equations, and optimization. He was also a lecturer in the departments of Physics and Mathematics at San Jose State University. Dr. Huynh obtained simultaneously a B.S. in Chemical Engineering and a B.A. in Applied Mathematics from UC Berkeley and an M.S. and a Ph.D. in Physics from UCLA. He is a member of INCOSE. References Ackoff, R. L., "The Future of Operational Research is Past," Journal of the Operational Research Society, Volume 30, 1979, in Critical Systems Thinking Directed Readings, R. L. Flood and M. C. Jackson (Editors), 1991. Ackoff, R. L., Ackoff's Best. His Classic Writings on Management, John Wiley & Sons, Inc., New York, 1999. Angel, C. D. and Froelich, J., Six Sigma: What Went Wrong?, 2008, http://www.destinationcrm.com/Articles/Columns-Departments/The-Tipping- Point/Six-Sigma-What-Went-Wrong-51394.aspx, last accessed 26 April 2009 ANSI/EIA-632, Processes for Engineering a System, American National Standards Institute and Electronics Industries Association, 1999. Arnold, S. (Editor), ISO 15288 Systems engineering — System life cycle processes, International Standards Organisation, 2002. Boehm, B., "Integrating Software Engineering and Systems Engineering", Systems Engineering, Vol. 1 (1994), no. 1. Bottomly, P. C., Brook, P., Morris, P. W. and Stevens, R., "A Study of the Relationship of Systems Engineering to Project Management", proceedings of Fourth Annual Symposium of the INCOSE-UK, 1998. Brekka, L. T., Picardal, C. and Vlay, G. J., "Integrated Application of Risk Management and Cost of Quality", proceedings of The 4th Annual International Symposium of the NCOSE, 1994. Brill, J. H., "Systems Engineering ---A Retrospective View", Systems Engineering, Vol. 1 (1998), no. 4, 258-266. Bruno, M. E. and Mar, B. W., "Management of the Systems Engineering Discipline", proceedings of the 7th Annual Symposium of the International Council on Systems Engineering, Los Angeles, CA., 1997. Chapanis, A., "Human Engineering," in Operations Research and Systems Engineering, C. D. Flagle, W. H. Huggins and R. H. Roy (Editors), Johns Hopkins Press, Baltimore, 1960. Denzler, D. and Kasser, J. E., "Designing Budget Tolerant Systems", proceedings of The 5th Annual International Symposium of the NCOSE, St Louis, MO., 1995. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 18 of 21

DOD 5000.2-R, "Mandatory Proceedures for Major Defense Acquisition Programs (MDAPS) and Major Automated Information System (MAIS) Acquisition Programs," US Department of Defense, 2002. Drucker, P. F., Management: Tasks, Responsibilities, Practices, Harper & Roe, New York, 1973. DSMC, Systems Engineering Management Guide, vol. May 1996, Defence Systems Management College, 1996. EIA 632, "EIA 632 Standard: Processes for engineering a system," 1994. Eisner, H., Computer Aided Systems Engineering, Prentice Hall, 1988. Eisner, H., Essentials of Project and Systems Engineering Management, John Wiley & Sons, Inc., 1997. Emes, M., Smith, A. and Cowper, D., "Confronting an identity crisis - How to brand systems engineering", Systems Engineering, Vol. 8 (2005), no. 2, 164-186. Fisher, J., "Defining the Roles and Responsibilities of the Systems Engineering Organization/Team", proceedings of the 6th Annual Symposium of the International Council on Systems Engineering, Boston, MA, 1996. Frank, M., "Knowledge, Abilities, Cognitive Characteristics and Behavioral Competences of Engineers with High Capacity for Engineering Systems Thinking (CEST)", The INCOSE Journal of Systems Engineering, Vol. 9 (2006), no. 2, 91-103. GAO, "DEFENSE ACQUISITIONS Major Weapon Systems Continue to Experience Cost and Schedule Problems under DOD’s Revised Policy," GAO, 2006. GDRC, The Problem Solving Process, 2009, * http://www.gdrc.org/decision/problem- solve.html, last accessed 11 Jan 2009 Gelbwaks, N. L., "AFSCM 375-5 as a Methodology for System Engineering", Systems Science and Cybernetics, IEEE Transactions on, Vol. 3 (1967), no. 1, 6-10. Goode, H. H. and Machol, R. E., Systems Engineering, McGraw-Hill, 1959. Hall, A. D., A Methodology for Systems Engineering, D. Van Nostrand Company Inc., Princeton, NJ, 1962. Hall, C. S. and Lindzey, G., Theories of Personality, John Wiley & Sons, 1957. Hammer, M. and Champy, J., Reengineering the Corporation, HarperCollins, New York, 1993. Hari, A., Weiss, M. and Zonnenshain, A., "ICDM - An Integrated Methodology for the Conceptual Design of New Systems", proceedings of System Engineering Test and Evaluation Conference SETE 2004, Adelaide, Australia, 2004. Hiremath, M., "Systems Engineering in Acquisition Strategy: Change Needed," INCOSE Insight, Vol. 11 (2008), no. 5, 32-33. Hitchins, D. K., Putting Systems to Work, John Wiley & Sons, Chichester, England, 1992. Hitchins, D. K., World Class Systems Engineering - the five layer Model, 2000, http://www.hitchins.net/5layer.html, last accessed 3 November 2006 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 19 of 21

Hitchins, D. K., Advanced Systems Thinking, Engineering and Management, Artech House, 2003. Honour, E. C. and Valerdi, R., "Advancing an Ontology for Systems Engineering to Allow Consistent Measurement", proceedings of Conference on Systems Engineering Research, Los Angeles, CA., 2006. Hudson, S. (Editor), Systems Engineering Competencies Framework, INCOSE UK, 2006. IEEE 1220, "Std 1220 IEEE Standard for Application and Management of the Systems Engineering Process," 1998. INCOSE Fellows, A Consensus of the INCOSE Fellows, 2009, http://www.incose.org/practice/fellowsconsensus.aspx, last accessed 18 March 2009 Jain, R. and Verma, D., "Proposing a Framework for a Reference Curriculum for a Graduate Program in Systems Engineering", INSIGHT, Vol. 10 (2007), no. 3, 9-11. JAXA, Basics of Systems Engineering (draft ) , Version 1B, 2007. Jenkins, G. M., "The Systems Approach," in Systems Behaviour, J. Beishon and G. Peters (Editors), Harper and Row, London, 1969, p. 82. Johnson, S. B., "Three Approaches to Big Technology: Operations Research, Systems Engineering, and Project Management", Technology and Culture, Vol. (1997), 891- 919. Kasser, J. E., "Systems Engineering: Myth or Reality", proceedings of The 6th International Symposium of the INCOSE, Boston, MA., 1996. Kasser, J. E., "Transition via Transactions: First steps in creating a customer driven organization", proceedings of First World Customer Service Congress, Tyson's Corner, VA, 1997. Kasser, J. E., "Writing Requirements for Flexible Systems", proceedings of INCOSE- UK Spring Symposium, 2001. Kasser, J. E., "The Cataract Methodology for Systems and Software Acquisition", proceedings of SETE 2002, Sydney Australia, 2002a. Kasser, J. E., "Systems Engineering: An Alternative Management Paradigm", proceedings of The Systems Engineering Test and Evaluation Conference, Sydney Australia, 2002b. Kasser, J. E., A Framework for Understanding Systems Engineering, Booksurge Ltd, 2007. Kasser, J. E. and Mackley, T., "Applying systems thinking and aligning it to systems engineering", proceedings of the 18th INCOSE International Symposium, Utrecht, Holland, 2008. Kasser, J. E. and Palmer, K., "Reducing and Managing Complexity by Changing the Boundaries of the System", proceedings of the Conference on Systems Engineering Research, Hoboken NJ, 2005. Martin, J. N., Systems Engineering Guidebook: A process for developing systems and products, CRC Press, 1997. Maslow, A. H., The Psychology of Science, Harper and Row, 1966. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 20 of 21

Maslow, A. H., Toward a Psychology of Being, Van Nostrand, 1968. Maslow, A. H., Motivation and Personality, Harper & Row, 1970. Microsoft, Transforming Education: Assessing and Teaching 21st Century Skills,, 2008, http://download.microsoft.com/download/6/E/9/6E9A7CA7-0DC4-4823-993E- A54D18C19F2E/Transformative%20Assessment.pdf, last accessed 20 March 2009 MIL-STD-490A, Specification Practices, United States Department of Defense, 1985. MIL-STD-499, Mil-STD-499 Systems Engineering Management, United States Department of Defense (USAF), 1969. MIL-STD-499A, Mil-STD-499A Engineering Management, United States Department of Defense (USAF), 1974. MIL-STD-499B, "Draft MIL-STD-499B Systems Engineering," United States Department of Defense, 1993. Murray, H. A., Explorations in Personality, Oxford University Press,, 1938. O’Toole, P., "Do's and Don'ts of Process Improvement", proceedings of 2005 U.S. Census Bureau SEPG Conference, Suitland. MD, 2004. OVAE, Problem-Solving Process, 2005, http://www.c- pal.net/course/module3/pdf/Week3_Lesson21.pdf, last accessed 11 Jan 2009 Pennell, L. W. and Knight, F. L., "Draft MIL-STD 499C Systems Engineering," The Aerospace Corporation, 2005. Peters, T. and Austin, N., A Passion for Excellence, Warner Books, 1985. Peters, T. J. and Waterman, H. R., In Search of EXCELLENCE, Harper and Row, 1982. PMI, A Guide to the Project Management Body of Knowledge (PMBOK Guide), Project Management Institute, 2004. Rodgers, T. J., Taylor, W. and Foreman, R., No-excuses Management, Doubleday, 1993. Roe, C. L., "The Role of the Project Manager in Systems Engineering", proceedings of The 5th Annual International Symposium of the NCOSE, 1995. Sheard, S. A., "Twelve Systems Engineering Roles", proceedings of The 6th Annual International Symposium of the NCOSE, Boston, MA., 1996. Taylor, F. W., The Principles of Scientific Management, Harper & Brothers Publishers, 1911. Todaro, R. C., "Lecture Handout, ENEE 648R," in, University of Maryland University College, 1988. Wasson, C. S., System Analysis, Design, and Development concepts, principles and practices, Wiley-Interscience, Hoboken, New JErsey, 2006. Watts, J. G. and Mar, B. W., "Important Skills and Knowledge to Include in Corporate Systems Engineering Training Programs", proceedings of The 7th Annual International Symposium of the INCOSE, Los Angeles, CA, 1997. Woolfolk, A. E., "Chapter 7 Cognitive views of learning," in Educational Psychology, Allyn and Bacon, Boston, 1998, pp. 244-283. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 21 of 21

Wymore, A. W., Model-Based Systems Engineering, CRC Press, Boca Raton, 1993.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 10

PLC Home Network System for Earthquake Disaster Prevention

Daisuke Honda Kyosuke Harayama Masahiro Inoue Shibaura Institute of Shibaura Institute of Shibaura Institute of Technology Technology Technology [email protected] [email protected]

Copyright © 2009 by Daisuke Honda, Kyosuke Harayama, and Masahiro Inoue. Published and used by APCOSE with permission.

Abstract. To reduce earthquake damage, Earthquake Early Warning started operating in Japan. The warning conveys the outbreak of earthquakes to home by broadcast and telecommunication. The warning gives home owners precautions several seconds before the earthquake wave reaches home. To utilize the information within limited seconds, home network system and controlling software are indispensable. In this study, we propose a system to reduce damage of the disaster by alarming family and controlling household appliances by Power Line Communication (PLC). We examine preventive means to reduce the congestion in PLC caused by interference among the adjacent houses during receiving Earthquake Early Warning. In addition, we investigate algorithm to determine the most suitable evacuating instruction and appliance controlling methods for each family depending on condition of each home. Introduction The earthquake has two kinds of waves. The Primary wave is high at speed, and the Secondary wave is slow at speed but large at amplitude which causes disaster. Earthquake Early Warning System consists of several processes; in the first process, the seismometers in each place detect Primary wave, and in second process, central computers calculate the location and magnitude of the earthquake and generate Earthquake Early Warning. The warning gives home owners precautions several seconds before the Secondary earthquake wave reaches home. To utilize the information within limited seconds, home network system (Inoue 2003) and controlling software are indispensable. HF band Power Line Communication (PLC) (Suzuki 2006), which is communication medium that does not need special wiring, is suitable for installing system in existing houses. However, interference and congestion may occur among adjacent houses when the traffic increases at time of disasters.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 10

Table 1: Specifications of HF band PLC Transmission Method Dispersed Tone Multi Modulation Method First Modulation Method DBPSK/DQPSK Access Method CSMA Transmission Speed 400,320,240,160,80kbps Frequency Band 2-9MHz

In this study, we propose reliable PLC system which reduces interference and expands communications distance in the PLC home network. In addition, we propose algorithm to decide the most suitable evacuating instruction and appliances controlling procedure for home depending on condition of each home. Requirements for PLC System The following are requirements for the high reliable information in the PLC home network. (1) PLC has interference among adjacent houses which may cause congestion and transmission delay. Preventive means are required to reduce PLC congestion during receiving Earthquake Early Warning. (2) Because there are various household appliances on the power line, they deteriorate PLC and shorten communications distance. Therefore, protocol which expands communications distance is necessary. We define specifications from the requirement mentioned above: z Communication delay in home network shall be less than 100msec. z Communication success rates without retransmission shall be more than 95%. PLC Home Network System We designed PLC system (figure 1 and 2) for apartment houses, where physical medium is shared by all apartments. When earthquake information is received, the central common transmitter in building broadcasts the first transmission to all appliances in building, and then each Home Gateway located at each apartment house multicasts the second transmission to the appliances at each apartment with Adaptive Delay-time Method to reduce interference.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 10

Figure 1. The use example of the Earthquake Early Warning at the apartment houses

Figure 2. PLC Home Network System for Earthquake Disaster Prevention

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 10

Reliable PLC Communication In the system, the second transmissions are multicasted with delay time depending on Signal to Noise Ratio (SNR) between the common transmitter and each gateway. The gateway whose SNR is lower can multicast second transmission with shorter delay time to back up first transmission.

The experiments were performed in the following procedures: (1) The common transmitter measured SNR between itself and each gateway by exchanging packets with each node (in non emergency time). (2) Second transmission delay time for each gateway was decided based on the measured SNR (Table 2) (3) All gateway nodes transmit packets with the delay time, and a monitoring node listened to all packets to examine the congestion and completion of communication. The deadline of all communication was decided as 100 msec in this system.. (4) It was examined whether Adaptable Delay-time Method to reduce interference based on Signal to Noise Ratio (SNR) in PLC can satisfy Requirements for PLC System.

Table 2: Adaptive Delay-time Method to reduce interference Communication SNR Delay time(msec) success rate SNR<40dB 0msec Under 100% 40

Transmission success rates were measured and compared between Interference condition where three nodes transmit at the same time and No Interference condition where each node transmits with Delay-time of 0,30,60msec. Reception success rate were measured.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 10

100% 90% 80% 70% 60% 50% 40% Interference Reception 30% success rate reception success rate(%) success reception

The first transmission and transmission first The 20% No interference 10% Reception success rate 0% 0 2 4 6 8 10 12 14 16 18 20 The traffic of the background node (kbps)

Figure 3. Reception success rate of PLC for secondly Warning

Figure 3 shows communication success rate comparing between with interference and no interference by adaptive delay. The reception success rate with no interference is 100% with background 18kpbs traffic. Even when background traffic increases to 20kbps, the reception success rate remains 96%. The experiment indicates Adaptable Delay-time Method to reduce interference based on Signal to Noise Ratio (SNR) in PLC is effective to improve communication success rate.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 10

Multi-hop PLC technique is also designed to cover wide communication range in home.

Figure 4. Delay time for multi-hop PLC

The total delay for multi-hop communication also shall be less than 100msec. Figure 4 shows that if packet size of Warning is 40-80 bytes, 3-4 hops will be acceptable to satisfy required completion time. Evacuating Notice Algorithm We investigate algorithm to determine the most suitable evacuating instruction and appliance controlling methods for each family depending on condition of each home. The evacuating instruction and appliance control are determined as function of the magnitude and time-to-arrive of earthquake wave, which in the Warning, characteristics of each building, and characteristics of resident (Table 3). The followings are evacuating notice procedure. (1) Estimate the damage of the house by the earthquake. (2) Decide evacuating notice from an estimate of damage and characteristics of resident(handicapped person, small child, aged person). (3) Improve the appropriate of instructions.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 10

Figure 5. Evacuating Notice Algorithm flow chart

Table 3: Contents comparison between conventional and proposed algorithm Conventional algorithm Proposed evacuating notice

algorithm Earthquake information Earthquake information and Evacuating notice evacuating instructions, contents household appliance control result Unavailable Characteristics of resident Consideration (handicapped person, small child, of resident aged person) Unavailable Characteristic of building Consideration (Structure, seismic of building intensity-resistant) The limit of the increase and Consideration of the increase and Consideration of decrease prediction of the seismic decrease of seismic intensity, and the ground intensity ground strength

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 10

We adopted Analytic Hierarchy Process AHP (Saaty 1990) (Expert choice) for evaluating evacuation notice algorithms. Figure 6 shows the AHP chart with criteria, priorities of criteria, and alterative. AHP analysis shows the proposed evacuating notice algorithm obtains priority of 0.667 and excels conventional algorithm which obtains priority of 0.333.

Figure 6. Analytic Hierarchy Process chart for evaluating evacuation notice algorithm

Table 4: AHP Sensitivity Analysis of comparison between conventional and proposed algorithm Priority Proposed Conventional evacuating notice algorithm algorithm Figure 6 0.333 0.667 In case of exchanging priority of "Giving priority to characteristics of family" for "Giving priority to 0.331 0.669 characteristic of the building" in Figure 6. In case of giving higher priority to "Household 0.332 0.668 appliance control" In case of giving higher priority to "Problem in the 0.502 0.498 system realization"

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 10

Conclusion To utilize Earthquake Early Warning, we propose the high reliable PLC home network system and the evacuating warning system for earthquake disaster prevention. To suppress the congestion on PLC caused by interference, the secondly Earthquake Early Warning in building is transmitted by Adaptable Delay-time Method depending on SNR. Multi-hop PLC technique is also designed to cover wide communication range in home. The maximum hops proved 3-4 to satisfy required completion time, if packet size of the warning is 40-80 bytes. The evacuating instruction and appliance control are determined as function of the magnitude and time-to-arrive of earthquake wave, characteristics of each building, and characteristics of resident. AHP analysis shows the proposed evacuating notice algorithm excels conventional algorithm. References Expert choice, http://www.expertchoice.com/ Inoue, Masahiro, Toshiyasu Higuma, Yoshiaki Ito, Noriyuki Kushiro, and Hitoshi Kubota. 2003. Network Architecture for Home Energy Management System. IEEE Transactions on Consumer Electronics. 49(3): 606-613. Japan Meteorological Agency, http://www.jma.go.jp/jma/en/Activities/eew.html Kang, Joon-Myung, Chang-Keun Park, Eun-Hee Kim, Jamees Won-Ki Hong, Yong-hun Lim, Seongho Ju, Moon-suk Choi, Bum-suk Lee, and Duckhwa Hyun. 2007. Design and Implementation of Network Management System for Power Line Communication Network. IEEE International Symposium on Power Line Communications and its Applications (ISPLC 2007):23-28. Liu, Er, Yangpo Gao, Golam Samdani, Omar Mukhtar, and Timo Korhonen. 2005. Powerline Communication over Special Systems, International Symposium on Power Line communications and Its Applications: 167-171. Saaty, T.L.1990. Analytic Hierarchy Process, RNS Publications. Suzuki, Kazumasa, Isamu Kawakami, Mamoru Sakugawa, Hiroyuki Kondo, and Hitoshi Kubota. 2006. High Frequency Band Dispersed-Tone Power Line Communication Modem for Networked Appliances. IEEE Transactions on Consumer Electronics, 52(1): 44-50.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 10

BIOGRAPHY Daisuke Honda is a graduate student, Shibaura Institute of Technology. He received his B.S. degree in Electrical Engineering and Computer Science from Shibaura Institute of Technology. He researched on evacuating notice algorithm using emergency earthquake information in Network Embedded System laboratory, College of Systems Engineering and Science, Shibaura Institute of Technology.

Kyosuke Harayama received his B.S. and M.S. degree in Electrical Engineering and Computer Science from Shibaura Institute of Technology. He researched on real-time network with PLC (Power Line Communication) in Network Embedded System laboratory, College of Systems Engineering, the Institute. He now works with SOFTBANK TELECOM Corporation.

Dr. Masahiro Inoue is a professor of College of Systems Engineering and Science, Shibaura Institute of Technology. His field of research includes embedded system, ubiquitous network, systems engineering, and project management. He received his B.S. and M.S degree in Applied Physics from Waseda University and Ph. D degree in Computer Science from Shizuoka University. He was engaged in research and development at Mitsubishi Electric Corporation. During 1990-1991, He worked at the University of Michigan as a visiting researcher. In 2005, He became a professor of Shibaura Institute of Technology. He is a Professional Engineer in Japan, a Project Management Professional.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 17

System Integration of Motorcycle Driving Stability Control Using SysML

Shaopeng Zhu, Hidekazu Nishimura Graduate School of System Design and Management, Keio University, 4-1-1 Hiyoshi, Kouhoku-ku, Yokohama-shi, Kanagawa-ken, 223-8526, Japan [email protected], [email protected] Laurent Balmelli, PhD. Manager, Guest Professor, Product Lifecycle Management Graduate School of System Design and Solutions, Management, IBM Software Group Keio University, Yokohama-shi, Japan [email protected] [email protected]

Copyright © 2009 by Shaopeng Zhu, Hidekazu Nishimura, Laurent Balmelli. Published and used by APCOSE with permission.

Abstract. In this paper, we investigate a system to control the stability of motorcycles during driving. In particular, the system helps for the prevention of so-called overturning accidents. We use the Systems Modeling Language (SysML) in order to systematically and efficiently develop the driving stability control system (DSCS). The DSCS contains a front-steering assist control system (FACS), an anti-lock braking system (ABS), and a traction control system (TCS). A conceptual design needs use cases of the DSCS. The DSCS’s role is to estimate disturbances to the motorcycle utilizing the motorcycle’s states and the rider’s operation, and accordingly actuate front steering, brake, and throttle using control signals. We use SysML to realize the use cases of the DSCS and clarify interactions and relationships among the DSCS and the associated external systems including a rider, existing vehicle systems, and road at Context Level. Moreover, we perform use case decomposition, structural decomposition, and logical operations at White Box Level. We explore several conceptual designs and devise an extended functional architecture for the DSCS, which in turn allow us to properly integrate the DSCS with the related external systems.

Introduction Motorcycle overturning accidents often happen due to disturbances of uneven road or slippery road in the transportation system where various vehicles and pedestrians come and go. In particular, the rider’s inappropriate operations largely influence the motorcycle motions and often cause the overturning accidents. Hence a driving stability control system to assist the rider’s operations is necessary. BMW Corp. developed a driving stability control system named Automatic Stability Control system and equipped it to commercial motorcycles. The Automatic Stability Control system integrated an anti-lock braking system (ABS) and a traction control system (TCS). It has been verified that the commercial motorcycle equipped with the Automatic Stability Control system has very good stability (http://www.motorcyclespecs.co.za/model/bmw/bmw_r1200r%2007.htm, http://www.webbikeworld.com/BMW-motorcycles/stability-control/). Meanwhile, Nishimura et al. equipped a front-steering assist mechanism with a servo motor to a motorcycle and performed a steady-state driving experiment. They verified that the front-steering assist mechanism could stabilize the attitude of the motorcycle by the driving experiment. Moreover, Nishimura et al. designed a front-steering assist control system using a model derived by system identification on the basis of the driving experiment (Kamata and Nishimura 2003, Kamata et al. 2003). In order to explore stability characteristics of motorcycles, the authors derived a dynamical model of a rider-motorcycle system taking account of rider’s movements. Using this dynamical model, we designed a front-steering assist control system to stabilize the motorcycle in steady-state straight driving and steady-state circular turning (Zhu et al. 2008, Iwamatsu et al. 2007, Zhu and Nishimura 2009). In this paper, we present a driving stability control system (DSCS) containing a front-steering assist control system (FACS), an ABS, and a TCS. In order to systematically and efficiently develop the DSCS, we use the Systems Modeling Language (SysML) (Balmelli 2007, Balmelli et al. 2005, Object Modeling Group 2006) to explore the conceptual design for the DSCS. At Context Level, we verify integration requirements between the DSCS and the associated external systems including a rider, existing vehicle systems and road. At White Box Level, we perform use case decomposition, structural decomposition, and logical operations. We explore several conceptual designs and devise an extended functional architecture for the DSCS, which in turn allow us to properly integrate the DSCS with the related external systems.

Driving Stability Control System Figure 1 shows a motorcycle equipped with the DSCS which contains a FACS, an ABS, and a TCS. We consider uneven road, slippery road, rider’s steering errors, etc., as disturbances that cause the unstable driving of a motorcycle in the transportation system. The DSCS’s role is to estimate disturbances to the motorcycle utilizing the motorcycle’s states and the rider’s operation, and accordingly actuate front steering, brake, and throttle using control signals.

Front-SteeringFrontFront- -Steering steering AttitudeAssist Control Control Driving Stability Control System + AntiAnti- -Lock lock Braking Brake System System + Traction Control System

Attitude ++ Speed Sensor Sensor

Servo ECU Motor Engine Wheel HU Wheel Sensor Sensor Brake Brake

DisturbanceDisturbances UnevenUneven ground road Small frictionSlippery coefficient road

Figure 1 A motorcycle equipped with driving stability control system Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 17

ABS and TCS The Automatic Stability Control system integrates an ABS and a TCS. The ABS can adjust brakes to prevent slipping of the motorcycle caused by wheel lock under braking condition. The TCS can control the throttle of the engine to present wheel spinning caused by losing tire traction

20 12 : With FACS 0 : Without FACS 10 8 -20 6 -40 4 -60 Y distance [m] distance Y 2 -80 [Nm] torque Disturbance 0 -100 -2 -100 -80 -60 -40 -20 0 20 0 2 4 6 8 10 X distance [m] Time [s] (a) Turning trajectory (b) Disturbance torque 2 15

0 10 -2

5

[deg] angle Roll -4 Steering angle [deg] Steering

-6 0 0 2 4 6 8 10 0 2 4 6 8 10 Time [s] Time [s] (c) Steering angle (d) Roll angle

4 30

20 2

10 0 0

Roll rate [deg/s] Roll Lean angle [deg] angle Lean -2 -10

-4 -20 0 2 4 6 8 10 0 2 4 6 8 10 Time [s] Time [s] (e) Lean angle of rider’s upper torso (f) Roll rate Figure 2 Comparison between simulation results of motorcycle applied impulse disturbance in steady-state circular turning with FACS and without FACS under driving condition (http://www.motorcyclespecs.co.za/model/bmw/bmw_r1200r%2007.htm, http://www.webbikeworld.com/BMW-motorcycles/stability-control/).

Front-steering Assist Control System (Kamata and Nishimura 2003, Zhu et al. 2008, Iwamatsu et al. 2007, Zhu and Nishimura 2009) When a motorcycle is subjected to some disturbances and a rider cannot stabilize the motorcycle in a moment, the FACS is useful to stabilize the motorcycle. The FACS can apply a front-steering assist torque generated by a servo motor to the steering shaft to assist rider’s steering. The experimental results have demonstrated the effectiveness for the attitude stabilization of the motorcycle in steady-state straight driving. Also the simulation results in steady-state circular turning shows good performance on the stabilization. Figure 2 shows the simulation results obtained by using a dynamical model taking account of leaning of the rider’s upper torso. In this simulation, when the motorcycle running at a speed of 30 km/h in steady-state circular turning, we applied an impulse disturbance torque of 10 Nm to the steering shaft as uneven road as shown in (b). In Figure 2, the blue solid lines show the simulation results with FACS, and the red dashed lines show those without control. (a) shows the turning trajectory. (c), (d), and (f) show the steering angle, the roll angle, and the roll rate of the motorcycle respectively. (e) shows the lean angle of the rider’s upper torso. It is seen from (c) to (f) that FACS suppresses the vibration of the steering angle, and reduces the amplitudes of the roll angle of the motorcycle by about 27 % and the lean angle of the rider’s upper torso by about 30 %. Moreover, FACS shortens the settling time of the roll angle by about half.

Conceptual Design for DSCS Using SysML To systematically and efficiently develop the DSCS, we adopt SysML to complete conceptual designs and an extended functional architecture of the DSCS. The SysML allows us to verify integration requirements between the DSCS and the associated external systems at Context Level. The SysML also allows us to concurrently find an extended functional architecture for the DSCS, while allocating the product functions to the chosen architectural elements at White Box Level.

Context Level When designing the DSCS, an important task is first to decide what belongs to the system being developed and what does not (Balmelli 2007). Three actors (i.e. associated external systems) for the DSCS are identified at Context Level: Existing Vehicle Systems (comprised of motorcycle electrical system, frame, steering shaft, wheels, brakes, throttle, etc.), Rider, and Road. Two basis use cases are defined at Context Level: Normal Driving Stability Control and Exceptional Operation. We represent the three actors and connect them to their respective use cases in Figure 3. In this figure, a rectangle is used to identify the boundaries of the DSCS. We describe two use cases of Normal Driving Stability Control and Exceptional Operation owned by the DSCS in the rectangle. In order to realize the DSCS design for higher reliability, we considered some attributes of the actors shown by in Figure 3, which may influence the design quality. It is seen that the external systems associated with the Normal Driving Stability Control use case are the Existing Vehicle Systems and the Road, and the external system associated with the Exceptional Operation use case is the Rider. Expressing of the behavior of a system equates to realizing its use cases under a specified set of non-functional constraints (Balmelli 2007). We express the behavior of the DSCS using sequence Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 17

Figure 3 Use case diagram of the DSCS

Figure 4 Sequence diagram for realizing the Normal Driving Stability Control use case diagrams to realize its use cases. In sequence diagrams, lifelines represent system instances, and incoming and outgoing messages represent interactions between the systems. A synchronous message (or an asynchronous message) is drawn by a line to the receiving system with a solid arrowhead (or a stick arrowhead). A return message is drawn by a dashed arrow with a stick head (http://www.ibm.com/developerworks/rational/library/3101.html).

Use Case Realization. At Context Level, we analyze interactions between the DSCS and the associated external systems (i.e. by DSCS Environment View). 1. Normal Driving Stability Control use case Figure 4 shows the sequential interactions used to realize the Normal Driving Stability Control use case. The sequence diagram describes these interactions: The DSCS requires the Existing Vehicle Systems to provide power. Meanwhile, the DSCS is required to start system, sense state, and perform optimal control by the Existing Vehicle Systems, and estimate disturbances by the Road. It is seen that a set of requirements for the DSCS

Figure 5 Sequence diagram for realizing the Exceptional Operation use case

Figure 6 Integration of the DSCS with the external actors at Context Level Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 17

(e.g. Start System, Sense State, Perform Optimal Control, Estimate Disturbance) were derived in this sequence diagram. 2. Exceptional Operation use case In Figure 5, The DSCS is required to manually stop system by the Rider. The Manually Stop System requirement for the DSCS is derived in this sequence diagram. In the SysML modeling environment, after performing the operation named Check-in Behavior to the plotted sequence diagrams, interfaces between the DSCS and the external actors can be created automatically.

Integration of DSCS with External Actors. The use cases are realized and the interfaces are created automatically as above. So below, we can integrate the DSCS with the external actors as shown in Figure 6. Figure 6 uses a solid line link the DSCS to the external actors to describe <>. The DSCS provides the I-Driving Stability Control System interface, and the Existing Motorcycle Systems, the Rider, and the Road use it. I-Driving Stability Control System interface contains five operations (i.e. functions): Perform Optimal Control, Sense State, Estimate Disturbance, Start System, and Manually Stop System (in the SysML modeling environment, describes an operation.). In addition, the Existing Motorcycle Systems provides the I-Existing Motorcycle Systems interface containing the Provide Power operation to the DSCS. We verify integration requirements that the operations are contained by the interfaces, and integrate the DSCS with the external actors using the interfaces in Figure 6. In order to find derived requirements for the DSCS, we perform the use case flow-down at Context Level. In our modeling environment, White Box Level is created automatically through the use case flow-down. Figure 7 shows the use case flow-down at Context Level. In this figure, the Normal Driving Stability Control use case is decomposed into four use cases: State System, Sense State, Perform Optimal Control, and Estimate Disturbance. The

Figure 7 Use case flow-down at Context Level Exceptional Operation use case is decomposed into Manually Stop System use case. Note that the generated use cases at White Box Level correspond to the operations of I-Driving Stability Control System interface provided by the DSCS. Hence, realizing the use cases at White Box Level equates to realizing the I-Driving Stability Control System interface provided by the DSCS.

White Box Level At White Box Level, we will explore some conceptual designs and an extended functional architecture of the DSCS (i.e. by Functional View). The SysML allows us to perform system architecting and functional allocation jointly at White Box Level.

Use Case Realization. 1. Start System use case In Figure 8, we choose ECU Control and Sensing State as functional sub-systems of the DSCS for realizing the Start System use case. The ECU Control sub-system performs self-check and then requires the Sensing State sub-system to start detection. It is seen that the Perform Self-check function is allocated to the ECU Control sub-system, and the Start Detection function is allocated to the Sensing State sub-system. 2. Sense State use case In Figure 9, we reuse the ECU Control and the Sensing State sub-systems for realizing the Sense State use case. This figure uses loop fragment in which operations are performed repeatedly. The Sensing State sub-system is required to sense vehicle attitude, vehicle speed, and wheel speed by ECU Control sub-system. After that, the Sensing State sub-system requires the ECU Control sub-system to process the detected state. We allocate the Sense Vehicle Attitude, Sense Vehicle Speed, and Sense Wheel Speed functions to the Sensing State sub-system, and allocate the Process Detected State function to the ECU Control sub-system.

Figure 8 Sequence diagram for realizing the Start System use case Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 17

Figure 9 Sequence diagram for realizing the Sense State use case

3. Perform Optimal Control use case In Figure 10, we reuse the ECU Control and the Sensing State sub-systems, and choose Servo Motor Actuation, Hydraulic Unit Adjustment, and Driving Torque Adjustment sub-systems for realizing the Perform Optimal Control use case. Initially, the ECU Control sub-system processes the detected state and judges the driving state. It is seen that besides the Process Detected state function, the Judge Driving State function also is allocated to the ECU Control sub-system. When the rider cannot stabilize the attitude of the motorcycle (i.e. when the roll angle of the motorcycle is beyond the decided safety range), the ECU Control sub-system requires the Servo Motor Actuation sub-system to generate an assist steering torque. Hence we allocate the Generate Assist Steering Torque function to the Servo Motor Actuation sub-system. The Motorcycle Steering Shaft external actor is required to utilize the assist steering torque for stabilizing the attitude of the motorcycle (i.e. FACS is realized.). When the slip rate is beyond the decided safety range, if the motorcycle is braking, the ECU Control sub-system requires the Hydraulic Unit Adjustment sub-system to adjust brake hydraulic pressure. We allocate the Adjust Brake Hydraulic Pressure function to the Hydraulic Unit Adjustment sub-system. After that, the Motorcycle Front/Rear Brakes external actor is required to utilize the brake pressure for preventing wheel lock (i.e. ABS is realized.). If the motorcycle is acceleration, the ECU Control sub-system requires the Driving Torque Adjustment sub-system to adjust throttle opening. We allocate the Adjust Throttle Opening function to the Driving Torque Adjustment sub-system. After that, the Motorcycle Front/Rear Wheels external actor is required to utilize the driving torque for preventing wheel spinning (i.e. TCS is realized.). e Perform Optimal Control use case Figure 10 Sequence diagram for realizing th

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 17

4. Estimate Disturbance use case In Figure 11, we reuse the ECU Control and the Sensing State sub-systems for realizing the Estimate Disturbance use case. The ECU Control sub-system processes the driving states detected and estimates the road surface state. Hence we allocate the Estimate Road Surface State function to the ECU Control sub-system besides the Process Detected State function. 5. Manually Stop System use case In Figure 12, we reuse the ECU Control sub-system and choose the Manual Operation sub-system for realizing the Manually Stop System use case. The Manual

Figure 11 Sequence diagram for realizing the Estimate Disturbance use case

Figure 12 Sequence diagram for realizing the Manually Stop System use case

Operation sub-system requires the ECU Control sub-system to stop the DSCS. We allocate the Deactivate System function to the ECU Control sub-system. Above all, it is seen that the expression of interactions between the sub-systems results in functions being allocated to the sub-systems. Note that we repeatedly use the ECU Control and the Sense State sub-systems to realize a set of use cases for the DSCS at the White Box Level. We perform again the operation named Check-in Behavior to the plotted sequence diagrams, interfaces among the sub-systems of the DSCS and the external actors (e.g. Motorcycle Steering Shaft, Motorcycle Front/Rear Brakes, and Motorcycle Front/Rear Wheels) are created automatically. We carry the requirements included by the interfaces to the next analysis level through the use case flow-down. Figure 13 shows the use case follow-down after two decomposition steps. In Figure 13, the use cases of White Box Level_01 Level are decomposed into more detailed use cases. The Start System use case is decomposed into Perform Self-check and Start Detection use cases. The Sense State use case is decomposed into Sense Wheel Speed, Sense Vehicle Speed, Sense Vehicle Attitude, and Process Detected State use cases. The Perform Optimal Control is decomposed into Judge Driving State, Generate Assist Steering Torque, Adjust Throttle Opening, and Adjust Brake Hydraulic Pressure, and Process Detected State use cases. The Estimate Disturbance use case is decomposed into Estimate Road Surface State and Process Detected State use cases. The Manually Stop System use case is decomposed into Deactivate System use case. We find that the Sense State use case, the Perform Optimal Control use case, and the Estimate Disturbance use case include the same use case Process Detected State though the use case flow-down.

Figure 13 Use case flow-down after two decomposition steps Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 13 of 17 Level (extended functional architecture) Figure 14 Integration chart at White Box

Extended Functional Architecture for the DSCS. The system architectural elements as sub-systems were chosen and the product functions were allocated to the chosen elements as above. Accordingly, we integrate the sub-systems of the DSCS to complete a functional architecture for the DSCS by using the created interfaces. In addition, we integrate the sub-systems of the DSCS with the associated external systems to complete an extended functional architecture. Figure 14 shows the extended functional architecture for the DSCS. In this figure, the notation presents a composite association path, the notation presents a containment path, the notation presents providing an interface, and the notation presents using an interface. In Figure 14, the DSCS contains six functional sub-functions: ECU Control, Servo Motor Actuation, Hydraulic Unit Adjustment, Driving Torque Adjustment, Sensing State, and Manual Operation. Moreover, the ECU Control is decomposed into Disturbance Estimation, Driving Stability Controller, and Driving State Judgement. The Sensing State is decomposed into Sensing Vehicle Attitude, Sensing Vehicle Speed, and Sensing Wheel Speed. We investigate some attributes of the sub-systems presented by the notation . For example, we investigate the rate of information, memory, and operating frequency, etc. for the ECU Control sub-system. The Servo Motor Actuation, the Hydraulic Unit Adjustment, the Driving Torque Adjustment, the Sensing State, and the Manual Operation sub-systems communicate with the central sub-system ECU Control using <>. The ECU Control sub-system uses these interfaces: I-Servo Motor Actuation, I-Hydraulic Unit Adjustment, I-Driving Torque Adjustment, I-ECU Control, and I-Sensing State. Meanwhile, the ECU Control sub-system also provides the I-ECU Control interface to the Sensing State and the Manual Operation sub-systems. The Servo Motor Actuation sub-system uses the I-Motorcycle Steering Shaft interface provided by the Motorcycle Steering Shaft external actor to realize the FACS. The Hydraulic Unit Adjustment sub-system uses the I-Motorcycle Front/Rear Brakes interface provided by the Motorcycle Front/Rear Brakes external actor to realize the ABS. The Driving Torque Adjustment sub-system uses the I-Motorcycle Front/Rear Wheels interface to realize the TCS. Accordingly, we properly integrate sub-systems of the DSCS and the associated external systems and complete the extended functional architecture. Conclusions In this paper, we applied the Systems Modeling Language (SysML) to the development of a driving stability control system (DSCS) through the application of model-driven methodology. In the modeling environment, we explored several conceptual designs and devised a functional architecture for the DSCS, which in turn allowed us to properly integrate the DSCS with the related external systems. The DSCS is composed of a front-steering assist control system (FACS), an anti-lock braking system (ABS) and a traction control system (TCS). This purpose of the system is to enhance the stability of the motorcycles, and prevent so-called “overturning accidents”. The role of DSCS lies in the estimation of disturbances depending on the motorcycle’s states and the rider’s operation. Then the system accordingly actuates the front steering, the brake, and the throttle using control signals. At Context Level, the key benefit of our model-based analysis is to allow us to verify integration requirements between the DSCS and the associated external systems. For that purpose, we defined three external actors: Rider, Existing Motorcycle Systems, and Road, and captured two usage scenarios, namely Normal Driving Stability Control and Exceptional Operation. We realized these use cases using sequence diagrams to express interactions between the DSCS and Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 15 of 17

the external actors. This operation allowed us to identify derived requirements, which we carried to the next analysis level through an operation named flow-down. At White Box Level, the key benefit of our model-based analysis is to allow us to concurrently find an architecture for the system, while allocating the product functions to the chosen architectural elements. When realizing sequence diagrams at White Box level, we chose functional elements as sub-systems, and expressed interactions between them. The expression of interactions between the sub-systems results in functions being allocated to the sub-systems. On the other word, system architecture and functional allocation are performed jointly. These interfaces resulting from the realization operation allow us to properly integrate sub-systems of the DSCS and the associated external systems while verifying and satisfying the initial product use cases.

References BMW R 1200R http://www.motorcyclespecs.co.za/model/bmw/bmw_r1200r%2007.htm BMW Motorcycle Traction Control. 2007. http://www.webbikeworld.com/BMW-motorcycles/stability-control/ Kamata, Y., and Nishimura, H. 2003. System Identification and Attitude Control of Motorcycle by Computer-Aided Dynamics Analysis, JSAE Review, Vol.24, No.4, pp.411-416. Kamata, Y., Nishimura, H., and Iida, H. 2003. System Identification and Front-Wheel Steering Control of Motorcycle (in Japanese). In Transactions of the Japan Society of Mechanical Engineers, Series C, Vol. 69, No. 688, pp. 3191-3197. Zhu, S., Nishimura, H., Iwamatsu, S., and Tajima, H. 2008. Dynamical Analysis of Motorcycle by Multibody Dynamics Approach. In Journal of System Design and Dynamics, Vol. 2, No. 3 Special Issue on Nonlinear Dynamics in Mechanical Systems, pp. 703-714. Iwamatsu, S., Nishimura, H., Zhu, S., and Tajima, H. 2007. Linearized Model and Motion Control of Motorcycle (in Japanese). In Proceedings of the 10th Symposium on Motion and Vibration Control, No. 07-13, pp. 186-191. Zhu, S., and Nishimura, H. 2009. An Attitude Stabilization Control System for a Motorcycle (A front-steering assist control for steady-state circular turning at the low speed), (in Japanese). In Transactions of the Japan Society of Mechanical Engineers, Vol. 75, No. 753 Balmelli, L., 2007. An Overview of the Systems Modeling Language for Products and Systems Development. In Journal of Object Technology, Vol. 6, No. 6, pp. 149-177. Balmelli, L., Brown, D., Cantor, M., and Mott, M. 2005. Model-Driven System Development. In IBM Systems Journal, Vol. 45, Issue 3, Special issue, pp. 569-585. Object Modeling Group 2006. The Systems Modeling Language 1.1 Specification, Document -- ptc/06-05-04 (SysML final adopted specification), http://www.omgsysml.org. Bell, D., Specialist, I., UML’s Sequence Diagram, http://www.ibm.com/developerworks/rational/library/3101.html.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 17 of 17 Biography Shaopeng Zhu is now a Ph.D. student of Graduate School of System Design and Management, Keio University under the supervision of Prof. Nishimura. Her research area includes dynamical analysis and motion control for mechanical systems such as a motorcycle. She is now interested in model-driven systems engineering and systematic design of products. She is a member of JSAE.

Hidekazu Nishimura is now a professor of Graduate School of System Design and Management, Keio University, Japan. His research area includes mechanical systems control and motion control for dynamical systems. He is now interested in model-driven systems engineering and systematic design of products. He is a member of INCOSE, IEEE, JSME and JSAE.

Laurent Balmelli is currently a manager at IBM in charge of architecting the new generation of offerings and tools for Systems Engineering and Product Development Since 2003, He has represented IBM within the SysML standard team and is one of the lead authors of the SysML language specification. He is also guest professor at Keio University in Tokyo Japan.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 14

Towards Better Adaptation of Systems Engineering To Japanese Space Program

Masashi Okada1, 2 and Masaru Nakano2 1Chief Engineer Office, Japan Aerospace Exploration Agency ２Graduate School of System Design and Management, Keio University [email protected]

Copyright © 2009 by Masashi Okada and Masaru Nakano. Published and used by APCOSE with permission.

Abstract. After a series of major failures of a launch vehicle and spacecraft missions in 2003, JAXA, which is the Japanese Space Agency, started to improve its technological bases and development process by using a methodology from systems engineering. Though these activities are still ongoing, we have already obtained some useful outcomes. On the other hand, there are many areas left to be reconsidered in order to apply systems engineering to the Japanese Space Program with taking advantage of the knowledge from Japanese manufacturing management systems that are often characterized as “integration-based manufacturing” compared to “modular-based manufacturing”. This paper presents the analysis of the problem-cause structure of the development process in JAXA to improve by deep understanding of the mission and systems engineering technology as a test case. Introduction In the 1960s, “systems engineering” was established in the United States in order to produce large and complex systems. It has successively evolved with relevant methodologies and penetrated all over the world according to each organizational culture. Today, many people recognize the definition as “an interdisciplinary approach and means to enable the realization of successful mission [INCOSE, 2007]”, or “a methodical, disciplined approach for the design, realization, technical management, operations, and retirement of a system [NASA, 2007]”. In Japan, the concept of “systems engineering” was introduced when launch vehicle technology was transferred from the United States at the beginning of the 1970s. Although it was well absorbed and adapted to the space development program by Japanese engineers in their own way at that time, its essential meaning has gradually been faded away due to continued successes of the space missions. As a result, its meaning and style have become ambiguous for the last thirty years. In 2003, three organizations of NASDA, ISAS and NAL, were merged into JAXA. It did not take long until JAXA experienced a series of major failures, including those of the Japanese main launch vehicle H-IIA, earth observation satellite MIDORI-II, and Mars orbiter NOZOMI. Each failure cause was investigated systematically and quantitatively, and countermeasures were taken including enhancement of reliability. The background and essential factors for the failures were analyzed, and an approach for improvement of space development process was studied in JAXA with newly established advisory board composed of both JAXA personnel and outside advisors including the international professionals from various space agencies. The “systems engineering” approaches were enhanced across various fields such as management

of technology, program and project management, discipline engineering and mission assurance. As a result, the entire development process was investigated to be restructured [JAXA, 2005]. Chief Engineer Office newly established in 2005 and directly connected to the top management of JAXA, firstly conducted a survey of international standards on project management and systems engineering [USAF, 1969; ANSI/EIA, 1999; IEEE, 2005; NASA, 2006; NASA, ECSS, 2004; NASA 2005, INCOSE, 2007] and a study to employ them in JAXA. Then they published “Basics of Systems Engineering” [JAXA, 2006] and started to penetrate the essential meaning into the organization. In 2007, new development processes were defined having compatibility with world-class standards with considering smooth transition from the current processes（see Figure 1）. Since then, many new missions were created and some of them were approved to be projects.

All the organizations and personnel related to project team shouldmakean effortforthe missionsuccess. Wrap-up and handover SAC Prior-evaluation SAC Prior-evaluation Knowledge accumulation JAXA Implementation Plan Approval JAXA Implementation Plan Completion Mission Proposer Pre-Project Manager Project Manager (incl. Working Group) Pre-Project Team Project Team dissolution of Definition Preparation Implementation Project Team Sharing perception about initial requirement, risk etc. Sharing perception about initial requirement, risk etc. Upper level management responses risks that exceed the Decision Making Decision Making responsibility and authority of project manager. by President Mission Definition by President Project Initiation Whole preparation activities Activities of whole project Change of scope All necessary resources All necessary resources Increase of budget Upper limit of project cost Termination Project Preparation Project Approval Review Validity of Mission Definition Review Validity of Project Initiation Problem occurence • Compliance with mid-term plan • Compliance with mid-term plan about accomplishment • Value of mission •Value of mission of mission • Realizability of overall plan • Realizability of overall plan (incl. human resource & budget) (incl. human resource & budget)

Project Support Check and Balance for Project

Proposal Proposal ‹Review ‹Independent Evaluation ‹Project Oversight Council ‹Management through Daily Report (Director in charge, S&MA, CEO) Development Completion Mission Definition Review System Requirement Review System Definition Review PDR CDR Review ・・・・・・Preliminary Final Production Concept Studies Concept Development Project Formulation Design Design and Testing Operations

Other requirements and constraints Contractor selection Project Manager Mission success criteria process ‹System Analysis and Definition z Responsibility and authority for supervising team activities z Project execution by best techniques and knowledge with reducing risks Mission requirement (draft) System requirement (draft) System spec. (draft)

Concept studies Concept development scope System feasibility System feasibility Total cost estimation ‹Communication with stakeholders Sharing of role ‹Project Planning ‹Total project management ( incl. risk, team performance etc. ） Project plan (draft) ‹Conducting of development & operation Higher level requirement Systems engineering plan (incl. review) ‹Requirement management ( incl. consistency & compatibility to requirement, countermeasures etc.)

• Project objectives • WBS and activities Risk identification and response • Scope definition • Life cycle cost schedule resource • Mission success criteria • Risk analysis/response ：Baseline within scope • Development policy • Precondition/constraints out of scope • Organization (draft) • Other relevant plan etc. Figure 1. Outline of development process in JAXA

However, results of interviews to the fourteen project managers who applied new development process indicated that there were still many problems left. Some problems are related to "efforts-meaning balance” and "approach of the entire organization" which are parts of systems engineering. In addition, the results indicated that it is not sufficient only to employ systems engineering in the organization with the following circumstances: Š Front-loading activities before project approval are enhanced; nevertheless, the resources (human and budget) were not sufficiently allocated afterwards.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 14

Š Documentation activities for review process dramatically increased since the system engineering approach was employed; therefore, genuine engineering activities were compressed. Š Reviewer's role and contents of the review was not defined clearly. Š Information, knowledge and experience from other projects were not sufficient. Š A guideline of systems engineering was just a paper without practices. Š The systems engineering department should support projects giving their feedback. In order to establish systems engineering in the organization, it is necessary to analyse current problems and to continue with improvement until it is completely established in the organizational culture. This paper provides an effective approach that realizes a good system by conveying voice of customers to manufacturing enterprises as a value chain. Firstly, the problems in the space program in Japan are drilled down. Secondly, the space program is discussed in a context of positioning in the viewpoint of government, industrial and cultural aspects. Furthermore we identify features which are similar to other highly matured industries in Japan such as automotive and electronics with a survey of recent studies on them. Thirdly, the authors present an approach to resolve the problem that often happens in employment of traditional systems engineering technology to Japanese organization like JAXA. The key to the approach is to focus on the upstream of lifecycle process. The conclusion includes the future research. Visualization and Focusing of Problem Structure In this section, in order to focus down the discussion points, we refer to some reports and results of interviews to engineers. Thereby the issues on Japanese space program are picked up and structured. Data. The undesirable results for the space program are roughly classified to following four categories; loss of meaning of the project, fatal trouble on orbit, cost overrun and schedule delay. These results are related mutually. When a project does not succeed, the project finishes as those balances have finally lost. Finally, the continual occurrence of these project failures results in “the nonfulfilment of long-term plan in space program”. The analysis reports concerning on-orbit malfunctions of spacecrafts [JAXA, 2004] and launch failure [JAXA, 2008] were surveyed so that the basic structure of possible background factors that cause the results mentioned above were obtained. As a result, the background factors were extracted. In order to improve the comprehensiveness of them, the additional keywords were complemented by interview to professionals inside the space agency. Method of Analysis. We used the concept of “Current Reality Tree” of “Theory of Constrains” which Goldratt originates [Goldratt, 2002; The Goldratt Institute, 2001], defined the interrelationship between each factor and attempted to overlook them. AHP (Analytic Hierarchy Process) [Saaty, 1992] was introduced here in order to quantify the relative level of the each factor th as expressed in Figure 2. Here, ak,n represents the magnitude of the n element in the level-k.

N ak,1 k-1 ak,n=Σ a(k-1),n n=1

ak-1,1 ak-1,2 ak-1,n

ak-2,1 ak-2,2

Figure 2. AHP application to current reality tree Results of Analysis. Figure 3 shows the result of analysis. Each factor is weighted using the method above mentioned, according to the reply from experienced engineers inside the space agency. The factors that are weighted correspondently by most engineers are highlighted in the figure.

Failure in long term plan

Loss in mission value

Malfunction in Delay of orbit / development Cost overrun Launch failure Unforeseen problem Many Delivery delay Invisibility of Requirement Low- Low- Lack of troubles in of progress change accuracy of accuracy of margin development subcontractors planning cost- concept estimation Long-term Mismatching of Mission project true needs postponement by Design Random Operation Variation of Lack of budgetary restraint error failure error product quality verification Long-time design, development and Delay of production CONOPS Difficulty of full Reduction of environmental verification Inadequate Rigidity of Lack of right- Difficulty of Difficulty of simulation reflection f rom requirement requirement first-time design quality control quality control engineering change for total quality f or other for overseas other malfunction organization parts

severe Too many Huge cost for Limited Decision lack of Less commu- Manufacturing special parts verification opportunity for delay for feasibility nication with parameter and process verification specification study stakeholders

Lack of Lack of Difficulty of Inadequate Dysfunction of criterion information domestic parts understanding of engineering acquisition phenomena and process modeling

Lack of Inadequate Losing substance Poor Lack of technological penetration of of process review consideration for assessment process to projects robustness and redundancy

Lack of understanding for process effectiveness Poor system Lack of Insufficient explicit thinking experience and knowledge knowledge High-context development >10% of total weight style Lack of consistency (ambiguity-integrated) from technological > 5% of total weight strategy to project Figure 3. Current reality tree of Space Program in Japan This approach provides the primary background factors and the relationship among them by capturing the problem structure widely and overlooking them. The results are shown as follows; Š Lack of communication with stakeholders and insufficient feasibility studies affect the quality of the requirement analysis. The main causes are likely that engineering process is

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 14

not penetrated to projects or it loses substance. In addition, lack of evaluation of technical application could be irrelevant to them. Š Design malfunction results from inadequate understanding and modelling of phenomenon. In addition, insufficiency of design review to the design could be irrelevant to them. Š Lack of the system thinking that considers robustness and the redundancy of systems. Š In the case of the project of which verification scale is large, pressure of the verification plan according to the decision delay or change of the specification makes the verification to be insufficient. These issues mentioned above are allocated on the development process by using the analyzing method of engineering process and inherent problem. [Kubota et al., 2006]. The timing in which the issues are embedded (white dot) and appear as the phenomenon (black dot) is described in Figure 4.

Figure 4. Engineering process and inherent problems

Focusing. It is beyond the scope of this paper to discuss the issues which corresponds to individual process. This paper describes that many factors mentioned above are embedded in the first half of lifecycle where effort of the space agency is relatively high and the results appear in the latter half of the life cycle where the role of contractors become larger. It should be emphasized that the main player in the process in which the problems are embedded is different from that of problems appearance.

Features of Space Development in Japan After the following chapter, the correspondence to issues is discussed while learning from the development system of the other matured industries. Firstly, in this chapter, the feature of the Japanese space program is made visible by positioning from industrial aspect and cultural one. Customer Value Chain. Figure 5 shows results of CVCA (Customer Value Chain Analysis) [Donaldson et al., 2006] concerning the relation among the stakeholders of the Japanese space program. These results could be a base for the following discussion. The Japanese space agency, as well as other foreign space agencies, contracts with the prime manufacturer on the national budget, provides the system and/or the service to the stakeholders and finally returns the result to the public. It is clear that it has a role which adds value at the uppermost stream of the value chain to realize customer needs while there is a difference due to characteristics of each program. Š Mission in which space agency provides to users （space applications etc.） Public Government International partner

Government user Space agency User Prime contractor

Š Mission in which space agency collaborates with community （space Subcontractor Public Government International partner

Space agency

Prime contractor Community

Subcontractor Š Mission in which space agency implements directly (Space station ) Public Government International partner

: Information Space agency : Money Prime contractor : Claim : System Subcontractor Figure 5. Customer value chain of space development in Japan

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 14

To understand in more detail, a value chain model is shown in Figure 6, which expresses each role realizing needs of the government user or the community as a system. It is found that the relationship between the space agency and the manufacturer (prime contractor and subcontractor) is close very much.

Government Space agency Enterprise Academia –Policy determination –Proposal to government –Proposal to government –Academic research Mid-, long-term plan from space agency's and space agency from –Advisory to government standpoint enterprise’s standpoint

–Research grant –Research with given –Research with internal –Collaborative research –Budgeting to space budget fund with space agency （Advanced research） agency –Research with external fund –Research grant –Strategic research ( in –Research under contract –Collaborative research –Budgeting to space house, collaborative or with space agency with space agency agency contract ) –Research with internal –Research with external Practical research –Analysis and definition fund fund of mission requirements –prior evaluation –Architecture design for –Engineering with a view –Cooperation as experts –Budgeting to space system requirements to production agency definition –Detailed design down Development –Project management to components level with a view to operation –Technical decision as experts –Project management –Production and quality Production with a view to operation assurance –Technical oversight and response as acquirer –Project management –Conducting validation with a view to operation on the ground Validation –Technical oversight and –Response to validation response as acquirer results –Procurement of launch –Providing launch service service –Technical oversight and Maintenance of Infrastructure Launch response as acquirer –Launch safety –Initial checkout on orbit –Validate of on-orbit Operation performance

–Utilization as –Utilization as space –Utilization as government user agency community –post evaluation –Promotion of utilization –Creation of academic Utilization –Yielding benefit –Yielding benefit research results towards public towards public –Yielding benefit towards public

Figure 6. Value chain model in detail

Industrial Position. A space system is required higher level in safety and reliability because the opportunity of the mission is very little and the impact to the society caused by failure is very large. The characteristics of the space program are compared with other industries using x-,y-axis concerning above-mentioned features. From viewpoint of vertical axis which shows the system safety and reliability, the space system belongs to the same category as an automobile. It is necessary for the automobile to be integrated optimally and to be adjusted by many parts and components delicately because a commercial value is enlarged by safety enhancement similar to comfort, silence and operability. As a result, it does not move on to “Modular type architecture [Fujimoto, 2003]”, but has “Integral type architecture [Fujimoto, 2003]”, [Nobeoka, 2006]. From viewpoint of horizontal axis which shows the opportunity of experience accumulation, the space system belongs to the same category as a nuclear power plant. It does not make much

difference from the automobile industry. However, both have common aspect as “a series of the processes which transcribe customer requirements to the material as design information [Fujimoto, 2003]”, in other words, the product is just the result of “manufacturing”.

System safety/ reliability high ●(initial) ●(present) ●Automobile Nuclear power plant ● ● Stove Accumulation of Space system repeated experiences

low high ●Special ●Home electric ● House ● Craftwork low • Ambiguous boundary between R&D • Clear differentiation between product and manufacturing development and merchandise development • Risk management in phased • Importance of manufacturing process planning project development • Intensive test production and testing

Figure7. Industrial position

Cultural Position. Hall has defined ”context” as ”the information that surrounds an event”, and observed that Japan is originally an extremely high context culture[1976]. The systems engineering is “the way to effective systems engineering management is not in the direction of formal, formidable, and massive documentation [INCOSE, 2007]”, on the other hand, it is still based on the documents and the management of them through the lifecycle. In the high context culture, they originally communicate with less document description. It can be the factor to disturb the penetration of systems engineering into Japanese organization, as appeared in the interviews to the project managers mentioned above. High-context culture Feature in communication Japanese Arab - It takes much time to establish implicit information in advance. It is difficult to communicate without Greek the information. Spanish - Minimum explicit information in conveyed Italian message English - Simple, effective and short-time communication French - Function that binds and unifies persons together - Long-lived, stable and effective information American Scandinavian German German-Swiss - Most information is included in conveyed message. - Less function that binds persons together Low-context culture - Short-lived and changeable information Figure8. Position in context culture

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 14

Survey of Literature Concerning Japanese Production System As mentioned above, the space system is realized by chained relationship among government users, communities, the space agency and manufacturers (prime contractor and subcontractor). In order to adjust systems engineering to Japan, it is valuable to refer the model of product development process which expresses the characteristic of Japan. Regarding the product development process, many empirical studies on that of Toyota have been performed. The pioneering study on Lean Thinking by Womack and Jones et al. [1996] has concluded that “the critical starting point for lean thinking is value, which can only be defined by the ultimate customer”. Furthermore, they have mentioned that Toyota Production System (TPS) is regarded as the lean production(manufacturing) system which achieves “a bigger result by using less resource in all respects” and also shows the five steps (Define Value, Identify the value stream, Make value stream flow, Let the customer pull products, Perfection) for execution. Herewith, acknowledgment and the introduction of the lean production system have been attempted centering on the automobile industries in Europe and the United States. Morgan and Liker [2006] have expanded the lean manufacturing system to the consistent product development system, which is consisted of thirteen principles based on the interrelated and interdependent subsystems of “Process, person, tool, and technology”. On the one hand, Fujimoto has stated that the manufacturing type of many Japanese industries which have grown up in postwar period is compatible with the products which have “integral architecture” in terms of the organizational ability of “integration-based manufacturing”. In addition, he has also mentioned that “product development” is creating the information of design and that the feature of the companies which accumulates the organizational ability surpassing other companies is “a skill of method of letting flow the information of design” through the total system – development, production and buying with the manufacturing system being considered based on the“information of the design”. Fujimoto has called total mechanism in which “the flow of design information towards customers” with teamwork as “integration-based manufacturing” [Fujimoto, 2005]. In particular, organizational ability of the product development aspect is mentioned as follows [1997]; (i) to increase prediction accuracy for uncertain market needs by shortening development period between planning and launching (ii) to enhance adaptation ability for the change and diversity of the market by improving productivity and increasing the number of projects under the given budget (iii) to enhance the component engineers’ understanding ability for product concept by organizing the small-numbered radical and tight project team and by taking the wide workable range of each engineer (iv) to assign the powerful leader (heavy-weight product manager) who is not only coordinator but also the responsible person for both concept creation and technical translation of concept. Recently, Lean Advancement Initiative of MIT has researched both systems engineering which is the activity at the upstream of system lifecycle and lean thinking which is occurred at downstream. Here, lean thinking is defined as “the dynamic, knowledge-driven and customer-focused process through which all people in a defined enterprise continuously eliminate waste with the goal of creating value”. They have recognized that both systems engineering and lean thinking have the common goal of delivering product or system lifecycle value to the customer, and have researched on the concept of ”Lean Systems Engineering” combined those two which realizes a superior

process [Rebentsch and Murman, 2004].

Discussions about Direction in Systems Engineering of Japanese Space Program One of the common challenges of the product development and the manufacturing system in these advanced studies is “how do we make “the information flow that creates the additional value for customers in chains ”by using the process, person and technology/tools” [Poppendieck, 2006] . In the space program of Japan, systems engineering was introduced by the United States, and engineers have groped better way of research and development under various restrictions (limited technological accumulation, human resource, budget and industrial size etc.). In particular, the space agency takes the contract for development of subsystem individually and integrates them as a system. Besides, engineers have created concepts and developed by integration-based way in a high context culture. The implementation of a project under such restrictions increases a specific risk, i.e., the process progression without a sufficient trade-off. The Japanese space program seems to have gotten over the risk by following an integration-based way. Today, roles of the space agency and industries are reviewed in order to enhance the reliability, in consequence, “prime contract type” for development and manufacturing becomes mainstream. In this case, the main body of the technological activity is taken over from space agency by the prime contractor at the point of phase transition from concept study(pre-phase A)/concept design(phase A) to preliminary design (phase-B). These series of process is shown in Figure 9. Here, the risk element that intrudes into each process is described in addition. The information from the space agency to the prime contractor is basically transmitted by documents. By thought of the history of Japanese systems engineering and the technological culture above mentioned, it is concerned that engineers feel more stress by documentation of information which creates additional value in chain.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 14

Space agency leads Contractor leads

Feedback

Defining Stakeholder Requirements Developing Feasible Design Solution Developing Preliminary ‚ Less communication with System stakeholders ‚ lack of feasibility study Requirements ‚ Mismatching of true needs ‚ Lack of technological ‚ Inadequate requirement assessment Acquisition engineering ‚ Poor system thinking ‚ Less consideration of Proposal restriction And Awards ‚ Misunderstanding of requirements ‚ Error of requirements flow-down ‚ Decision delay for Developing a specification System-of- Interest that Meets Acquirer ‚ Value chain break

‚ Error of requirements flow-down ‚ Inadequate understanding of phenomena and modeling ‚ Decision delay for specification Figure 9. Process flow in phase transition (Referred notation: [Lambert, J. et al., 2006]) Considering the information flow (information transmission and sharing process) which create additional value from the viewpoint of customers to be made smooth, the directionality of systems engineering in the space program of Japan is shown as follows. Strong leadership connecting customer to the manufacturing site: The manager in the space agency is a so-called spokesperson of the customer value, and also the person that charges an integrated responsibility creating concept, manufacturing and bringing value to customers. The manager of the prime contractor is a person that shares value as a partner of the space agency and penetrates the value into the prime contractor and the supplier (chief engineer's concept in the lean product development system). Both strong binding of both persons and sharing value among them realize smooth flow of value information. Combination of integration-based way and risk management: A high-level mission concept is drafted by communications with the stakeholders. The drafted concept is refined according to the integration-based process. It is important to define the process itself and to share it. In this process, it is necessary not to make a system decision early, and to optimize the interpretation of the concept while repeating many times (Concept of integration-based manufacturing). When such a non-deterministic process form is taken, it is

necessary to manage the risk to vagueness intensively. Simultaneous engineering which bridges between partners: There are some limits to complete the set of requirements and to transmit them deterministically to downstream process (lower-level systems engineers or manufacturing engineers etc.). Therefore, it is important to cooperate with persons engaged in downstream process (concept of simultaneous engineering). Collocation among the space agency, the prime contractor and the supplier is also effective measures. The concept of transmission and communications of the requirement is shown in Figure 10 as an example. Important points of the requirement transmission are sharing requirements without discrepancy when the process in the downstream starts and complying with the requirement from the upstream process when the downstream process is completed.

z CASE1: To enhance management of requirements using effective tools and to hand over highly qualified documents to downstream process. High-context Context Meaning Transition

Requirement definition Information Space agency Concept Concept Design Study Low-context - Capability of system realization including Prime Contractor Preliminary Design manufacturing (Candidate) - Pursuance of requirement quality Contractor - Complete requirement management Selection - Organizational risk management

z CASE2: To transfer the information by the context that supports the requirement management and to confirm each other by simultaneous effort of upstream and down stream. High-context Context Transition Meaning

Requirement definition Space agency Concept Concept Design Co-location Information Study Low-context Prime Contractor Concept Design Preliminary Design (Candidate) Concept Design - Sharing context by front-loading - Concept-leaded management rather than complete requirement management Contractor - Individual and exhaustive risk management Selection Figure 10. Concept of requirement relay and communication

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 13 of 14

Conclusions and Future Works This paper proposes an approach for adaptation of systems engineering to space program of Japan by considering industrial and cultural aspects and by focusing on the transition from concept formulation phase to product development phase. A future work is considered to specify systems engineering process fitting to Japan by comparing with other industries and evaluating the value chain quantitatively. We will consider broad issues for the product development system such as process, technological accumulation, personnel training, and the tools and techniques. Specifically, partnership between acquirer and supplier, contract type, the organizational philosophy and the ability are focused on. References INCOSE 2007 systems engineering handbook - a guide for system life cycle processes and activities, Version 3.1, ed. Cecilia Haskins.: 2.1.of 10. NASA. 2007. NASA/SP-2007-6105 Rev1: Systems engineering handbook: 3. JAXA. 2005. Advisory Commission for Mission Success Final Report, JAXA Press release. USAF. 1969. MIL-STD-499: Systems Engineering Management. Government Electronics and Information. 1999. ANSI/EIA-632: GEIA STANDARD Processes for engineering a system. IEEE Computer Society. 2005. IEEE Std 1220TM-2005: IEEE, Standard for application and management of the systems engineering process. NASA. 2006. NPR 7123.1: Systems engineering processes and requirements. European Cooperation for Space Standardization. 2004. ECSS-E-10 Part 1B: ECSS, Space engineering, system engineering- Part1: Requirements and process. NASA. 2005. NPR 7120.5C: Program and project management processes and requirements. JAXA. 2006. BDB-06007: Basics of Systems Engineering JAXA. 2008. Status of malfunction analysis and investigation on on-orbit satellites, Report to Space , Report to Space Activities Commission. JAXA. 2004. Investigation status of Committee for Fundamental Issue on Space Development, Report to Space Activities Commission. Goldratt, E. M. 2002. It's Not Luck (Japanese version), DIAMOND Inc., Japan. The Goldratt Institute, White paper: Theory of constraints and its thinking processes - A brief introduction to TOC. http://www.goldratt.com/asiapacific/toctpwpjapanese.pdf. Saaty, T. L. 1992. The Analytic Hierarchy Process. McGraw Hill, 1980. Reprinted as Vol.1 of the AHP Series by RWS Publications. Kubota, F Wada, K Araki, M Nakano, M Ochiai, T Shimada, K. 2006. Framework and tool for visualizing and analyzing vague processes during process reengineering, 2006 International Symposium on Flexible Automation (ISFA): 842-849. Donaldson, K Ishii、K and Sheppard, S. D. 2006. Customer Value Chain Analysis, Research in Engineering Design, Springer London, Volume 16, Number 4,pp174-183. Nobeoka, K. 2006. Introduction to MOT, Nikkei Publishing Inc., Japan. Fujimoto, T. 2003. Competing to Be Really, Really Good, Chuo Koron-shinsha Inc., Japan. Hall, E. T. 1976. Beyond Culture (Japanese version), Anchor Press/Doubleday, New yolk. Womack, J. Jones, D. and Roos, D. 1996. The Machine That Changed the World, Rawson Associates.

Womack, J and Jones, D. 1996. Lean Thinking , Simon and Schuster. Morgan, J. M. and Liker, J.K. 2006. The Toyota Product Development System: Integrating People, Process And Technology, Productivity Press. Fujimoto, T. 2005. MMRC-J-24: A note on comparative advantage of architectures, MMRC Discussion Paper Series, Manufacturing Management Research Center, University of Tokyo. Fujimoto, T. 1997. Evolution Theory of Manufacturing system, Yuhikaku Publishing Co., Ltd. Japan. Rebentisch, E. Rhodes, D.H. and Murman, E. 2004. Lean systems engineering: Research initiatives in support of a new paradigm, Publication / Presentation at the Conference on Systems Engineering Research. Poppendieck, M. &T. 2006. Implementing Lean Software Development: From Concept to Cash, Addison-Wesley Professional. Lambert, J. Jennings, R.K. Joshi, N.N. 2006. Integration of risk identification with business process models, Systems Engineering, Volume 9 Issue 3:187-198

ACKNOWLEDGMENT

This work was supported in part by Grant in Aid for the Global Center of Excellence Program for "Center for Education and Research of Symbiotic, Safe and Secure System Design" from the Ministry of Education, Culture, Sport, and Technology in Japan.

BIOGRAPHY Masashi Okada is a systems engineer of JAXA. He is responsible for development of systems engineering process, management of strategic technology roadmaps at the agency level, establishment of knowledge database and independent assessment of the current projects in JAXA, as one of the start-up members of Chief Engineer Office (CEO). CEO was established in October 2005 in order to recover JAXA from a series of mission failures in 2003 by improving the engineering capability of JAXA. Most part of his career started in 1989 was a launch vehicle engineer. He was engaged in hot firing tests of the liquid rocket booster engine, which was one of the key subsystems of H-II launch vehicle capable of 4 tons to GTO, system design of the following H-IIA and a conceptual study of next generation launch vehicle. He received a master’s degree in aeronautics and astronautics in 1986 from the University of Tokyo. Now, he takes a doctoral course at Graduate School of System Design and Management, Keio University and he is also a research assistant of Global COE Program“Centre for Education and Research of Symbiotic, Safe and Secure System Design”.

Masaru Nakano is a professor of Graduate School of System Design and Management, Keio University, Japan. He was a principal researcher and laboratory manager at Toyota Central R&D Labs., Inc before joining Keio. He received a B.S. and M.S. from Kyoto University and a Dr. Eng. from Nagoya Institute of Technology. Dr. Nakano has studied in the fields of robotics, manufacturing system design, enterprise integration, business process reengineering, marketing research and sustainable manufacturing. He is a member of IFIP, JSME, ORSJ, JIMA, and etc.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 10

A study on Interface Management based on Model

Sung Gyun OH, Young Won Park, Chan Mook Kim Department of Systems Engineering, Graduate School of Ajou University, Republic of Korea [email protected], [email protected], [email protected]

Copyright © 2009 Sung Gyun OH, Young Won Park, Chan Mook Kim. Published and used by APCOSE with permission

Abstract. As modern systems require a higher capability, it is characterized by complexity based on interfaces among systems. These characteristics make it difficult to manage the system interface. Moreover, if it fails, it is highly likely to result in system failures. This study provides a model-based system engineering environment for facilitated interface management activities by Integrated Product Team and deals with system interface identification and control through the case of maglev train.

Introduction

Purpose. The systems nowadays are evolved upon large and complex ; which means it is increasing a number of internal/external interfaces that is of target system, therefore managing interfaces become complex. Managing system interface is very important to develop system architecture. The inappropriate management of system internal/external interface may cause system failures. As mentioned above, interface of developing system gets complicated; hence it needs more attention for management. Especially, the case of large system, which is composed of many sub systems, often has different contractors for each system in order to minimize the risk. CASysE tool is capable to help communication among different organizations and support the interface management efficiently. Therefore, the aim of this study is to provide the environment for the purpose of efficient interface management for department of integrated product development team by CAsysE tool, and through the Malgev train case study, we examine its effects on large system’s interface identification and control.

Case Study: Maglev. This paper shows the case study of maglev train system Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 10

development. Since the architecture of train system is highly mature, it is possible to reuse existing architecture for new system development. Therefore, it is desirable to develop the maglev train system architecture referring to existing train architecture through reengineering process. In this way, we can reduce the efforts and cost for initial development stage.

Figure 1 Reusing architecture of reference systems

shows a diagram of maglev train system’s architecture that was reused existing train system’s architecture. In this figure, as previous train converted to maglev train, related 6 systems have been modified accordingly. Because the maglev train system works by collaboration function among sub systems, it is needed to have thorough management on interface issues that are caused by changes of each sub system. With respect to this reason, the maglev train system is a good case of system interface control.

System engineering process. In general, it is possible to find the precedent cases for system development except few cases. When we reuse the architecture of referral system, it can help to reduce the amount of tasks. This is main purpose of reengineering. Sage gave a definition of ‘product reengineering’ as followed; “ A product reengineering not only focuses on achieving the main purposes of existing products but also restructures new function and non-function by examining internal mechanism or function of existing product in order to apply new technology”. According to Sage’s definition, reengineering follows process sequence from forward engineering, reverse engineering and finally reengineering. A diagram of reengineering process is shown in

.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 10

Figure 2 System reengineering process

Reengineering process first distinguishes ‘Need’ for initial stakeholders and concept of system, and then performs ‘requirements definition process’ which is a process in forward engineering. Moreover, it performs a reverse engineering process by analyzing referral system in order to figure out system requirements. Lastly, these deduced requirements and generated requirements are merged together and then, performed ‘solution definition process’ of reengineering. The interface for performing reengineering process is for; z Achieving the interface relationship of either system of referral system or sub- component in reverse engineering process. z Being recognized by solution definition process for target system. This interface is managed by CASysE Tool.

CASysE Tool. A system engineering support tool provides a powerful development environment. To achieve the simultaneous development environment for many developers and mutual application among models from tool, it needs to determine the relationship among data components and the schema that defines the properties of components.

shows the schema for interface recognition and control. It modifies a Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 10

base schema of Vitech’s Core accordingly for application.

Figure 3 Schema of Interface control

We used 3SL Co.’s Cradle as a tool.

Interface Identification

Select Reference System. We choose the system that is closest to our operation concept. The maglev train system has most similar operation concept of LRT. We can deduce the architecture of physic-function-requirements sequentially by reverse engineering process of reengineering process.

Interface Identification of LRT System. We can identify the system interface and physical hierarchical structure through the documents related to OCD, System spec., URD, ICD and IDD of LRT system that selected by reference system. LRT is composed of 7 sub-systems as shown in

. When the physical hierarchical structure of system is identified precisely, the interface among these structures becomes definite. The identified interface, except those will be modified or not be affected by adding physical components and become non-necessary, mostly remains in the new system. After drawing the function by physical hierarchical structure, it is merged with requirements from forward engineering and then, performs reengineering process.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 10

Figure 4 Physical Hierarchy of LRT System

is a diagram of physical hierarchical structure that is refined through reengineering process. Although we cannot observe the changes on the Lv.1,, we can see the modified function/physical hierarchical structure which is caused by integrated requirements as we examine the sub-level. The area marked with red color at each system block indicates the changes of physical components against reference system. A width of area is proportional to the amount of modified components and also amount of system internal/external interfaces which are affected by components changes.

Figure 5 Physical Hierarchy of Maglev System

The interface that is identified from above mentioned process is put in interface form shown in

. This interface form contains; z Detail explanation of interface name, scope and function of interface, z Explanation of interface components z Interface relationship among components

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 10

Figure 6 Interface Form of Cradle

With respect to Thomas U.Pimmler and Steven D.Eppinger, we define the scope of interface among components in order to apply to DSM as follows;

z Energy exchange z Information exchange z Material exchange z Physical space & array

Moreover, we allow adding score in terms of importance of interface between components. Clustering that can be performed by DSM offers robust system integrated analysis. However, this study does not consider this and be limited to providing input form that is for the identified interface through templates.

shows a picture of Query View’s input data. By Query View, it is able to check the input-output relationship among components.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 10

Figure 7 Interface Query View

Interface Control

When the interface that is obtained by reverse engineering and reengineering process is built in database, ICWG is structured. The purpose of ICWG is to solve the interface problem which is difficult to solve by relationship of engineer versus engineer.

gives an interface management process. ICWG investigates and modifies requirements of interface so as to raise its maturity. Moreover, it is able to manage the impact on interface which is caused by components modification.

Figure 8 Interface Management Process Flow Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 10

This study referred to ‘guide to writing ICD’ from Department of Justice Systems Development Life Cycle Guidance Document. Proposed ICD structure is as followed.

Table 1 DOJ ICD No. Name Description 1 Scope System interface relationship/system definition 2 Concept of Operations Interface related operation scenario technique 3 Detailed Interface Generating interface requirements through No.2 Requirements 4 Qualification Methods Qualification methods for interface requirements of No.3 5 Notes 6 Appendices 7 Approvals Statement of interface approvals 8 Records of Changes Statement of records changes

The ICD in this document defines the scope of system (or components) that has an interface, and describes operation scenario. Furthermore, it shows interface requirements by recognizing the relationship between two systems’ (or components) interface, and encourages to state about validation requirements through operation scenario. It is able to present trackback relationship among components by Query View. Figure 9 shows Query View of interface requirements.

Figure 9 Interface Trace Query View

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 10

ICD that is suggested in this document has to present the history of changes. This study is able to check the issue of related ICD through issue that is shown in

.

Figure 10 Interface Issue Query View

Conclusions

In this study, we proposed the system interface recognition and control management by MBSE. In detail, we applied to “ Cradle ” under the reengineering development environment in order to be able to manage the system interface. The effect has been proved through the case study of maglev train. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 10

References

A.Sage and W. Rouse. 1998. Handbook of Systems Engineering and Management. John Wiley & Sons,INC. Electronic Industries Alliance(EIA). 1998. Process for Engineering a System(EIA-632). EIA Press. Arlington James N. Martin. 1997. Systems Engineering Guidebook: A Process for Developing Systems and Products. CRC Press. Boca Raton U.S. Department of Defense. 2001. Military Handbook: Configuration Management Guidance. Washington, DC: Systems Engineering Office The Department of Justice. 2003. Appendix C-16 Interface Control Document. Systems Development Lifecycle Guidance Document Tyson R. Browning, 2001. Applying the Design Structure Matrix to System Decomposition and Integration Problems : A Review and New Directions. IEEE Transactions on Engineering Management. Vol.48. No.48 : 294 Charles S. Wasson. 2006. System Analysis, Design, and Development. A John Wiley & Sons, Inc. Vitech Corp., CORE® User Reference Guide Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 10

Study on Interface Management based on Model

Sung Gyun OH, Young Won Park, Chan Mook Kim Department of Systems Engineering, Graduate School of Ajou University, Republic of Korea [email protected], [email protected], [email protected] Copyright © 2008 by Sung Gyun OH, Young Won Park, Chan Mook Kim Published and used by APCOSE with permission.

Abstract. As modern system requires higher capability, it is characterized by complexity based on interfaces among systems. These characteristics make it difficult to manage system interface, and if it fails, it is highly likely to result in system failures. The study provided model-based system engineering environment for facilitated interface management activities by Integrated Product Team and deals with system interface identification and control through the case of MAGLEV. Introduction

Purpose. 현대 시스템은 규모가 점차 커지고, 하부구성품 간의 상관 관계가 매 우 복잡해지는 방향으로 진화한다. 이는 대상 시스템 내/외부의 인터페이스 개수 가 점차 증가하고, 그로 인한 인터페이스 관리가 복잡해짐을 의미한다. 시스템 인터페이스의 관리는 시스템 아키텍처 개발활동에 있어서 중요한 요소 이다. 시스템의 내/외부 인터페이스에 대한 적절치 못한 관리활동은 시스템 실패 를 유발할 수 있다. 문제는 앞서 기술한 바와 같이 개발 시스템의 인터페이스의 복잡성이 점차 커지고 있기 때문에 이에 대하여 주의깊은 관리가 필요하다. 특히, 다수 시스템의 조합으로 이루어지는 대형 시스템의 경우 위험부담을 줄 이기 위해서 대체로 시스템 별 Contractor가 상이하게 존재하는 경우가 많다. 서 로 다른 조직간의 의사소통을 돕고, 이들의 인터페이스 관리 활동을 효과적으로 수행할 수 있도록 지원하기 위해서 CASysE도구의 적용이 마땅히 필요하다. 본 연구의 목적은 CAsysE도구를 통해서 통합재품개발팀이 인터페이스 관리활 동을 원활하게 할 수 있도록 환경을 제공하는데 있으며, 전형적인 대형 시스템인 자기부상열차에 대한 인터페이스 식별 및 통제에 대하여 적용하여 효과성을 확인 한다.

Case Study:Maglev. 본 논문에서는 자기부상열차 시스템의 개발사례를 예시로 한다. 자기부상열차시스템은 차량이 선로에서부터 일정 간격을 부상하여 주행하 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 10

는데, 시스템의 운영개념이 기존 철도 시스템과 유사하다. 철도시스템은 아키텍처 의 성숙도가 높기 때문에, 새로운 시스템을 개발할 때 기존의 아키텍처를 재사용 하는 것이 가능하다. 따라서, 자기부상열차는 초기 개발에 들어가는 노력과 비용 을 줄이기 위해서 재공학 프로세스를 통해 기존의 철도 아키텍처를 참조 시스템 으로하여 시스템 아키텍처를 개발하는 것이 바람직하다.

Figure 1 reusing architecture of reference systems

Figure 1은 기존 열차시스템 아키텍처를 재사용한 자기부상열차 시스템 아키텍 처를 도식화 한 것이다. 이 그림은 차량이 자기부상열차로 변경되면서, 이것과 인 터페이스 관계가 있는 다른 여섯 시스템의 크고작은 수정이 이루어진 것을 보여 준다. 자기부상열차시스템은 하부 시스템간 협동기능을 통해서 요구되는 임무를 수행 하기 때문에, 각 하부시스템의 변경으로 인해 발생되는 인터페이스 이슈에 대하 여 철저한 관리가 필요하다. 이러한 이유로 인해서, 자기부상열차 시스템은 인터 페이스 통제 사례로서 매우 바람직하다.

System engineering process. 일반적으로, 극히 일부 시스템의 개발을 제외하면 개발 선례가 되는 시스템을 찾을 수 있다. 참조시스템의 아키텍처를 변경하여 재 사용하면 상당한 수준의 업무 부담을 줄일 수 있다. 이것을 목적으로 하는 것이 재공학이다. Sage는 제품 재공학에 대하여 “제품 재공학은 기존 제품의 본질적인 주 목적 은 달성하면서 새로운 기술을 적용하기 위하여, 기존 제품의 내부 메커니즘 또는 기능을 검사, 연구, 식별 및 변형함으로써, 새로운 기능 및 비기능 형상으로 재구 성함을 말한다”라고 정의하였다. Sage의 정의에 의하면, 재공학은 순공학(forward engineering),역공학(Reverse engineering), 재공학(Reengineering)의 순서를 가지 는 제품 프로세스를 의미한다. Figure 2은 재공학 프로세스를 도식화 한 것이다.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 10

Figure 2 System re-engineering process

재공학 프로세스는 초기 이해관계자의 소요(Need)와 시스템 개념을 식별하여 요 구사항 정의프로세스를 수행하는 순공학과정을 수행한다. 그리고, 참조시스템을 분석하여 시스템 요구사항을 도출하는 역공학과정을 수행한다. 마지막으로, 이렇 게 도출된 요구사항과 앞서 생성한 요구사항을 통합하여 다시 해결방안 정의프로 세스를 수행하는 재공학과정을 수행하게된다. 재공학 프로세스를 수행하면서 인터페이스는; z 역공학 과정에서 참조시스템의 시스템 또는 하부 구성품간 인터페이스 관 계를 얻고, z 재공학 과정에서 대상시스템에 대한 해결방안정의 프로세스를 통해 식별 된다. 이렇게 재공학프로세스를 수행하면서 식별된 인터페이스는 CASysE Tool을 통해서 관리된다.

CASysE Tool. 시스템엔지니어링 지원 도구는 강력한 개발환경을 제공한다. 도구 가 제공하는 다수 개발자의 동시 개발 환경과 모델 간 상호운용성을 확보하기 위 해서는 데이터 요소와 그들간의 관계, 그리고 요소의 속성을 정의한 스키마의 정 의가 필요하다. Figure 3은 본 연구에서 인터페이스 식별 및 통제를 위해 사용하고 있는 스키 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 10

마를 나타낸다. Vitech사의 Core에서 제공하는 기본 스키마를 필요에 맞게 변경 하여 사용한다.

Figure 3 Schema of Interface controlt

본 연구에서는 도구로 3SL Co.의 Cradle를 사용하였다.

Interface Identification

Select Reference System. 고객 및 이해관계자로부터 정의된 소요와 개념을 통해 서 가장 유사한 운영개념을 가진 시스템을 선택한다. 자기부상열차 시스템은 경 량전철시스템(LRT)과 가장 유사한 운영개념을 가진다. 선택된 시스템은 참조시스 템으로써 재공학프로세스의 역공학 과정을 통해서 물리-기능-요구사항 아키텍처 를 순차적으로 도출된다.

Interface Identification of LRT System. 참조시스템으로 선택된 경량전철시스템의 OCD, System Spec., URD, ICD, IDD 등의 관련 문서를 통해서, 시스템의 인터페이 스와 물리계층구조를 확인할 수 있다. LRT는 아래의 Figure 4와 같이 7개의 하부 시스템으로 구성되어 있다. 이들 시스템의 물리계층구조가 명확하게 식별되면서 이들간의 인터페이스가 명확해진다. 식별된 인터페이스는 재공학 과정에서 변경 거나 추가될 물리구성요소로 인해서 영향을 받지 않는 것과 불필요하게 된 것을 제외하면 새로운 시스템에서 유지된다. 이후, 물리계층구조를 통해서 기능을 도출한 후, 최종적으로 시스템 요구사항을 식별하게되면, 순공학 과정을 통해서 얻은 원요구사항과 통합하여 재공학 과정을 수행하게된다. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 10

Figure 4 Physical Hierarchy of LRT System

Figure 5는 재공학 과정을 통해서 정제된 물리계층구조를 도식화한 것이다. Lv.1수준에서는 그 변화를 알 수 없으나 하부수준으로 가면서 통합된 요구사항으 로 인해 변경된 기능/물리계층구조를 확인할 수 있다. 각 시스템 블록에서 빨간색 으로 표시된 영역은 참조시스템 대비 물리구성요소의 변화를 의미한다. 영역이 넓이는 변경된 구성요소의 양과 비례한다. 또한, 영역의 넓이는 구성요소의 변경 으로 인해 영향을 받는(따라서, ICWG의 검토가 필요한) 시스템 내/외부 인터페이 스의 양과 비례한다.

Figure 5 Physical Hierarchy of Maglev System

위에서 기술한 과정을 통해서 식별된 인터페이스는 Figre 6의 인터페이스 폼에 입력한다. 이 인터페이스 폼에는; z 인터페이스의 이름과 인터페이스의 범위와 기능을 명시한 상세설명, z 인터페이스 대상 구성요소의 설명과 z 구성요소간 인터페이스 관계 의 내용을 포함된다.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 10

Figure 6 Interface Form of Cradle

본 연구에서는 DSM의 적용을 위해서 구성요소간 인터페이스 범주를 Thomas U. Pimmler와 Steven D. Eppinger가 제안하는 다음 4가지로 정의하였다.

z 에너지의 교환, z 정보의 교환, z 물질의 교환, z 물리적 공간 및 배열

그리고, 구성요소간 인터페이스 중요성에 따라서 점수를 가점할 수 있도록 하였 다. DSM을 통해서 수행할 수 있는 클러스터링은 강력한 시스템 통합 분석 기능 을 제공한다. 다만, 본 연구에서는 이것을 다루지 않으며, 템플릿을 통해서 식별 된 인터페이스를 입력할 수 있는 입력폼을 제공하는 것으로 한정하였다. Figure 7은 입력된 데이터를 Query View로 불러온 그림이다. 다음의 Query View를 통해서, 구성요소간 입출력관계에 대하여 확인이 가능하다.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 10

Figure 7 Interface Query View

Interface Control

역공학 과정과 재공학과정을 통해서 얻은 인터페이스가 데이터베이스에 구축이 되면, ICWG가 조직된다. ICWG의 목적은 단순한 엔지니어 대 엔지니어의 관계로 는 해결 하기 힘든 인터페이스 문제를 해결하는 것이다. Figure 8은 인터페이스 관리 프로세스에 대해서 나타내고 있다. ICWG는 식별된 인터페이스 요구사항을 검토 및 개정하여 그 성숙도를 높이는 업무활동을 한다. 그리고, 구성요소의 변경 으로 인하여 발생될 수 있는 인터페이스 영향을 검토하여 해결하는 활동을 수행 한다.

Figure 8 Interface Management Process Flow Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 10

본 연구는 ICWG의 인터페이스 통제활동을 지원하기 뷰를 제공함에 있어서, Department of Justice Systems Development Life Cycle Guidance Document의 ICD 작성지침을 참고하였다. 이 문서에서 제시하는 ICD의 구조는 다음과 같다.

Table 1 DOJ ICD No. Name Description 1 Scope 시스템 인터페이스관계/시스템 정의 2 Concept of Operations 인터페이스 관련 운용시나리오 기술 3 Detailed Interface 2장을 통해서 인터페이스 요구사항 생성 Requirements 4 Qualification Methods 3장의 인터페이스 요구사항 검증방법 기술 5 Notes 6 Appendics 7 Approvals 본 문서에서 식별된 인터페이스 승인자 명시 8 Records of Changes 본 문서의 변경 이력 기술

이 문서에서 제시되는 ICD는 인터페이스가 있는 시스템(또는 구성요소)의 범위 를 정해주고, 이에 대한 운용 시나리오를 기술한다. 그리고 운용시나리오를 통해 서 두 시스템(또는 구성요소)의 인터페이스 관계를 식별하여 인터페이스 요구사 항을 도출하고, 이에 대한 검증요구사항을 명시하도록 하고 있다. 본 연구는 아 래의 쿼리뷰를 통해서 앞서 기술한 요소간 추적관계를 현시할 수 있게 하였다. Figure 9는 인터페이스 요구사항 쿼리뷰이다.

Figure 9 Interface Trace Query View

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 10

또한, 이 문서에서 제시되는 ICD는 변경 이력을 명시하도록 하고 있다. 본 연구 는 Figure 10과 같이 Issue를 통해서 관련 ICD에 제기된 Issue를 확인 할 수 있 다.

Figure 10 Interface Issue Query View

Conclusions

우리는 MBSE 적용을 통한 시스템 인터페이스의 식별 및 제어 방안을 제시하 였다. 보다 상세하게, 우리는 재공학 개발환경에서 시스템 인터페이스의 관리가 가능하도록 Cradle에 적용하였으며, 그 효과성을 자기부상열차 인터페이스 식별 및 통제 사례를 통해서 확인하였다.

References

A.Sage and W. Rouse. 1998. Handbook of Systems Engineering and Management. John Wiley & Sons,INC. Electronic Industries Alliance(EIA). 1998. Process for Engineering a System(EIA-632). EIA Press. Arlington James N. Martin. 1997. Systems Engineering Guidebook: A Process for Developing Systems and Products. CRC Press. Boca Raton U.S. Department of Defense. 2001. Military Handbook: Configuration Management Guidance. Washington, DC: Systems Engineering Office The Department of Justice. 2003. Appendix C-16 Interface Control Document. Systems Development Lifecycle Guidance Document Tyson R. Browning, 2001. Applying the Design Structure Matrix to System Decomposition and Integration Problems : A Review and New Directions. IEEE Transactions on Engineering Management. Vol.48. No.48 : 294 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 10

Charles S. Wasson. 2006. System Analysis, Design, and Development. A John Wiley & Sons, Inc. Vitech Corp., CORE® User Reference Guide Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 13

On Systems Engineering Application Model for Korean National R&D Projects in Railway Systems Domain Yo-Chul Choi ([email protected]) SEKOREA Dongsun BD #801-1, 1141 Gwangjeong-dong, Gunpo-si, Gyeonggi-do, Rep of KOREA 435-805 Jae-Chon Lee ([email protected]) Department of Systems Engineering, AJOU University Woncheon-dong, Suwon-si Yeongtong-gu, Gyeonggi-do, Rep of KOREA 443-749

Copyright © 2009 by Yo-Chul Choi. Published and used by APCOSE with permission.

Abstract. This paper is concerned with the development of the Systems Engineering (SE) Application Model which is intended to be used for the Korean National Research & Development (R&D) Projects in railway systems domain. To construct the SE Application Model, we first studied a variety of the SE processes presented in the international and industrial standards. Then we tailored them to be fit to our purpose, that is, to be used in successfully directing the projects of our concern. The developed SE application model consists of the life cycle stages model for the R&D projects in railway systems domain and also two detailed application models to support SE activities at each life cycle stage. The two application models are made up of five core processes and are called the SE-based Project Management Model and the Systems Engineering Accomplishment Model, respectively. To assess how effective the developed the SE Application Model is in directing the projects, a process of the model evaluation using EIA/IS 731.1 standard (SECM; Systems Engineering Capability Model) has been performed. So far, the adoption of the developed model has been useful in connection with the project management and applying SE methods for the ongoing Korean National R&D Projects in railway systems domain. It is also noted that the model has contributed in upgrading the SE capability of the projects participants. It is hoped that the model concept in the paper will also be useful in other systems domains such as defense, aerospace, ship-building and so forth.

Introduction

According to the results of the latest studies, the most of the Korean national public development projects and research & development (R&D) projects have not been successful because of the planed budget overrun and the schedule delay and also sometimes the failure to meet performance target of the project. Case studies have indicated that a common cause is due to the incompleteness of the project planning in the early stages. There are opinions that a root cause could be the absence of applying the proper systems engineering expertise [1]. The systems engineering has attracted great attention in the area of successfully developing complex and large-scale products while satisfying faithfully stakeholder/customer requirements since late 1950s. Specifically, the early adoption of the SE process in the United States of America has been concentrated in defense and military systems domain. However, the application of SE has been expanded to in a variety of industrial domains including aerospace since the INCOSE founded in early 1990s has played a leading role in SE activities of the INCOSE. As a result, these days there are a lot of SE standards and guides available for many application domains [2]. In Korea, system development projects

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 13

have been carried out using the systems engineering methodology in railway, military, aerospace domains, etc. However, the application of SE process has been limited to some activities such as those of requirement analysis or verification of components development [3]. In particular, there has been no effort reported to be made in the Korean national R&D projects for which an SE model for the project including the life cycle is defined and the project is directed based on it [4]. The objective of the paper is to discuss how to develop the Systems Engineering application model for the Korean National R&D projects which can be applied directly in railway systems domain. To develop the model, we tailored the systems engineering process presented in various standards for SE process such as in ISO/IEC 15288(2002)[5], EIA 632[6], IEEE 1220[7]. Also, the detailed procedure and tools together with the environment are described to realize the model in practice.

Development of the Systems Engineering Application Model

1. Enhancement of Life Cycle Model for the Korean National R&D Projects

The existing life cycle model of the national research and development project in railway systems domain has been defined a relatively short planning study and basic design stages, the execution stages of national research and development has spent quite a little times. In case those requirements of stakeholders were not analyzed clearly because of a short planning study, a upright solution cannot be derived in basic design. Out of this products (or system) of project may be effected seriously. And despite there are a long term projects in railway systems domain, Life cycle of a research and development is very roughly defined and further assessment of a research and development of project has been performed manage mental aspects rather than technical ones by only special agency. The regulations related to management of the National R&D Project published in 2005 stress the importance of planning and assessment of stages, but does not describe in detail [8] To solve and improve problems, first of all, the life cycle model focus on projects of the railway systems domain is redefined as illustrated in Figure 1 below.

Figure 1: The improved Life Cycle Model in railway systems domain

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 13

2. Structure Development for Systems Engineering Application Model

The structure of the Systems Engineering Application Model for the National R&D project in focusing on the railway systems domain proposed in this paper shows same as illustrated in Figure 2 and 3 using process representation format of the EIA 632 standard. The developed Systems Engineering Application Model is based on the life cycle model for the National R&D Project, and this consists of two detailed application model; the SE Based Project Management Model and Systems Engineering Accomplishment Model.

Figure 2: Systems Engineering Application Model for the National R&D projects

These models are able to independently use as the scale and goals of the national research and development projects. These two models are made up of the five core process each and each process is described in sequence by stages; goal, tool, input, output, process details, process application schema, process computerization, and an auto-documentation of process implementation results. Especially we present the systems engineering process for the national R&D project using an IDEF0 modeling notation to do concretely and as specific as possible. The IDEF0 modeling notation can well express information (ICOM; input, output, control mechanism) of key activities in any process and show relationship and data between activities. In this study Partial key processes and main contents of this application model (SE Based Project Management Process, SE Based SW Development Process, Product Verification Process, Interface Control and Management Process, and Peer-Review Process) are showed by detailed IDEF0 diagrams.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 13

Figure 3: The proposed SE Application Model using EIA 632 Process Format

javascript:flink(%22the%22);The proposed model emphasized the importance of complementary cooperation between project management and systems engineering activities in the national R&D project. The ten processes as above are mapped down on the proposed life cycle model as illustrated in Figure 4 below. As a result, the systems engineering activities of the national R&D project follow life cycle model proposed can be possible for project organization to plan, accomplish, control, verify, document. Especially key results; SEMP, SR Based WBS, etc; are presented every phase of life cycle model. A key point is that the internal and external assessments are done by general project agency (team or group). Product Verification Process has done continuously in the R&D Project Execution and Verification phase, and final products of sub-project are assessed using a cross checking method by a general project agency.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 13

Figure 4: Mapping of the developed SE Application Process to National R&D Life Cycle stages

2.1 SE Activity Definition Process

SE activity definition process means that a process develops the SEMP (Systems Engineering Management Plan) that the overall systems engineering process deliverable is needed to well organize a project, and to support methods and product to be satisfied customer requirement. In particular, in case one large project has a lot of sub-projects as the national research and development project. Works to develop many SEMP and integrate them by one document are very difficult and spend much effort. In this paper, there show a general SEMP development procedure and computerization and auto-documentation methods to edit and print it immediately. The essential prerequisites in case of developing the SEMP to complete for the national research and development shall be identified lots of elements; project target and requirements, characteristics of project teams and members, management skill of a project organization, technology level, project support environment, project strategy etc.

2.2 SE Based Project Management Process

SE Based Project Management Process means that a process consistently establishes total project works based system requirements using a general project management method and tool as illustrated in Figure 5 below. Systems requirements are settled by a requirement analysis process in project design phase. The purposes of this process solve in advance problems of lack of

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 13

traceability can be happened between systems requirement and project work and support a progress of the project can be carry out without a hitch.

Figure 5: The detailed SE Based Project Management Process

2.3 Requirement and Architecture Application Process

Requirement and Architecture Application Process means that a process describes the procedure that transforms RFP (Request For Proposal) as source requirement to system requirements and defines a functional architecture and physical architecture using graphical method (notation) as DFD (Data Flow Diagram), FFBD (Function Flow Block Diagram), PAD (Physical Architecture Diagram) etc. and creates a system specification is necessary for system development and sub-system design. The purposes of this process would like to prepare criteria to accomplish total project objectives as there assigns definitely top level requirements of project to functions of sub-projects.

2.4 Requirement Based Project Implement Process

Requirement Based Project Implement Process means that a process reports and monitors the progress status of results during the detail project implementation which based the SE Based Project Management Process by WBS (Work Breakdown Structure) prepared through system requirements. The purposes of this process do not control sub-project activities but check in person

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 13

progress status of sub-project as objectives and system requirements of project by sub-project responsible team, and share a progress status between stakeholder of project, and supply system to solve in advance the unexpected problems.

2.5 SE Based Software Development Process

SE Based SW Development Process helps the ordering agency to support and manage the development of software for requirements related to software among the confirmed system requirements as illustrated in Figure 6 below. In the past the software development process leaves much to be desired at the collection and analysis of customer requirements. Especially a definition of system which consists of hardware and software is insufficient often as with architecture development efforts. It finally did not meet the customer requirements or often used to produce the product that did not satisfy design specifications for software. To solve this problem, we have defined software requirements at the beginning of the software development using the systems engineering techniques, requirement analysis and traceability, and developed the SOW (State of Work) which is based on software requirements. It intends to utilize them as the top documents of software development. Additionally this process can effectively support document verification activities during the software development.

Figure 6: The detailed SE Based SW Development Process

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 13

2.6 Product Verification Process

Product Verification Process means that a process performs verification of product which produced under project or final stage of the national research and development execution as illustrated in Figure 7 below. Product verification activities performed informally by exterior assessment agency in the end of project as in the past have been taken not a few problems in efficiency. On this study, it establishes systematically that the process of verification procedures of products and detailed activities and methods to perform effectively and efficiently verification process.

Verification Sub-system Decomposition Strategy Criteria (Defect Diagnosis) Decision Info. of Constraint of Verification Law/Regulation/Standard Info. Historical data Verification System Design Environment And Given Customer Requirements Report Life cycle Risk mgmt Info Guideline (Schedule Plan) information Establish a System Verification Verification Requirements Strategy Strategy A1 Selected targets and methods Master Verification Plan Prepare a Verification Plan Sub-project Verification Plan Verification Plan Verification Targets and Method list of Verification Sub-Verification Plan A2 Plan Verification Procedures Identify and notify constraints Verification Constraints

A3 Check a Data input and Develop a auto- traceability creation Auto-documentation Template Verification in MBSE Tools print document Enabling system status reports Readiness Template A9 A4 A8 Facility, equipment, operator status Verification Activities Results Reports reports related to enabling system Perform a Verificaton Collective Action Reports Activities Verification Readiness Reports Verification A5 Reports Sub-project Verification Reports Make Verification data valuable

Inconsistence factors to A6 Req. & Design Errors Technical Reports. Analyze and Mgmt Plan Useful data record and report verification Error Info. Useful data informations A7 Risk in service

Support MBSE Tools General Sub Verification TFT Facility/Equipment Test tools (Verification DB) Verification Team

Figure 7: The detailed diagram of the Product Verification Process

2.7 Interface Control and Management Process

Interface(IF) Control and Management Process is that it controls and manages the potential interface requirements which are not identify easily among sub-projects or systems and sub-systems at the requirement analysis stage as illustrated in Figure 8 below. The derived interface requirements through the Interface Control and Management Process can be used to define interface among sub-projects and to identify input & put of interface elements. It forms the foundation of solving unexpectedly problems of overlapping between sub-projects during the project.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 13

Figure 8: The detailed IF Control and Management Process

2.8 Risk Management Process

Most projects are executed in an environment of uncertainty. Uncertainty influences the ability of the project team to achieve the project objectives. Uncertainty includes events that could harm the project and those that could help the project. Risks are events that if they occur can jeopardize the successful completion of the project. To manage systematically risks is very important and risk should be identified and assessed for probability of occurrence and impact on the project. The purpose of the Risk Management Process is to identify, assess, and manage of all risks; technical engineering risk, project risk (cost, schedule, time etc). This paper presents that process, activities, organization, computer aided risk management, auto-documentation etc of risks

2.9 Configuration and Change Management Process

The purpose of Configuration and Change Management Process is to establish and maintain the control of requirements, architecture, and artifacts produced throughout the project(system)'s life cycle. Also it concludes information management produced throughout project(system)'s life cycle. The change is inevitable and managing the impact of change on a project is the work of Configuration management. Especially Configuration Management is to manage a change request, reviews, baseline management, and configuration documents according the project policy and plan. And Change management is to record and continuously monitor history for changing data and information.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 13

2.10 Peer-Review Application Process

At periodic intervals, development groups will want to double check the technology and others involved in a project via what is called a technical review. The technical reviews may be from a top to a low level review of the technical elements (requirement, design, specification, etc), some small walkthroughs or review by a technical peer. In this paper it is proposed that the Peer-Review Application Process informally carried out by internal procedure and format of project organization as illustrated in Figure 9 below. The purpose of the Peer-Review Application Process is to review a request for proposal, system requirement, project constraints, system architecture, formal documents, etc earlier than the formal technical review and deliver its results to the technical review. This process can identify and issue problems prior to the technical review, and then preponderantly review in the technical review. As results, the time of the technical review is saved considerably and not a few arguments can be removed among attendance.

Figure 9: The detailed Peer-Review Application Process

Assessment of the Developed Systems Engineering Application Model

In this paper we have performed the model evaluation using EIA/IS 731.1 standard (SECM; Systems Engineering Capability Model) to verify whether the developed the Systems Engineering Application Model for the National R&D Project in railway systems domain is effective or not. [9] The evaluation is performed to SE technology, SE management, and SE environment in the

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 13

EIA/IS 731.1 standard. Table 1 shows that assessment results of this model by SECM that leaves a systems engineering tool out of consideration. As a result, it shows accomplishment of 60% over to all specific practical activities from 1st level to 4th level, accomplishment of 80% over to 1st level and 2nd level. According to EIA/IS 731.1, the capability maturity level is recognized in case of accomplishment of 80% over each assessment criteria. Consequently the capability maturity level to this SE Application Model is 2nd level over and has the potential capability maturity to support a 3, 4, or 5 level.

Table 1: The Assessment results for the Model by SECM

Capability Maturity Level Category Focus Area 1 2 3 4 5 SP Acted SP Acted SP Acted SP Acted SP Acted No. SP No. No. SP No. No. SP No. No. SP No. No. SP No.

FA 1.1 Define Stakeholder and 5 5 7 5 5 5 1 1 0 0 System Level Requirements FA 1.2 Define Technical 12 11 11 10 2 1 0 0 0 SE Problem Technology FA 1.3 Define Solution 5 4 6 4 10 9 3 3 1 0 FA 1.4 Assess and select 5 5 5 5 4 2 0 0 0 0 FA 1.5 Integrate System 7 7 4 3 5 4 4 4 1 0 FA 1.6 Verify System 5 5 9 8 5 5 0 0 0 0 FA 2.1 Plan and Organize 9 9 16 15 8 8 0 0 0 0 FA 2.3 Integrate Disciplines 3 3 4 4 6 4 1 1 0 0 SE FA 2.5 Manage Risk 4 2 7 4 13 10 2 1 0 0 Management FA 2.6 Manage Data 5 4 5 5 3 1 0 0 0 0 FA 2.7 Manage Configurations 4 2 4 2 2 1 0 0 0 0 SE FA 3.1 Define and Improve the 1 1 7 3 22 7 8 4 1 1 Environment Systems Engineering Process Total 65 58 85 68 85 57 19 14 3 1 Achievement Rate (%) 89％ 80% 67% 74% 33%

CONCLUSIONS

The objective of the paper is to develop the Systems Engineering Application Model for the Korean National Research & Development (R&D) Projects which can be applied in railway systems domain. The effort has been made to define the SE Application Model and SE Key Activities that are required to plan, manage and verify effectively the progress from the beginning throughout the end in carrying out the projects. To achieve the goal, we developed a tailored life cycle model for the projects and the detailed systems engineering process that has to be applied at every phase of the life cycles. The features of the developed SE application model can be described as follows. First of all, the application model has been devised from a particular point of view, that is, by considering an appropriate life cycle model for the Korean National R&D projects

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 12 of 13

in railway systems domain. Second, the detailed system engineering process for each phase of the life cycle stages has been proposed to support this application model. Third, the developed model has been assessed based on EIA/IS 731.1 standard. Also, note that the model has been used in the ongoing Korean National R&D projects in railway systems domain to evaluate the effectiveness. As a result, the capability level of the proposed model in the paper is 2-level over. If a systems engineering tool is used, it can be leveled up to 3 or 4 over. The proposed model consists of the SE Based Project Management Model and Systems Engineering Accomplishment Model. To express the systems engineering process under study, an IDEF0 modeling notation is used to be more specific in model description. In addition, all the data and documents related to the projects are managed by a systems engineering tool. In conclusion, the proposed model has been useful in directing the Korean National R&D projects in railway systems domain in such a way that both the project organizations/teams and members can get help from the model while carrying out the systems engineering activities; SE Based project management, technical review, risk management, Peer-Review, Configuration Management and Change Control, SE-Based SW development, etc. It is hoped that the model concept in the paper will also be useful in other systems domains such as defense, aerospace, ship-building and so forth.

REFERENCES 1) Y.W. Park, “Preface of Systems Engineering Process (EIA-632 Korean language version)” SETechnology, 2006. 2) EIA, "EIA-632, Processes for Engineering a System", EIA Standard, pp 1~2, Jan. 1999. 3) S.H, Choi and three others, “A systems engineering application and reliability analysis of the Korean High Speed Railway”, KRRI, pp 65~98, Aug. 2006. 4) Y.C, Choi, “Systems Engineering Application Model for the National R&D Project (Focusing on the Railway system domain), A doctoral dissertation of Ajou University, Feb. 2007. 5) MOT, “The regulations related to management of the National R&D Projects”, a Presidential decree No 18731, Mar, 2005. 6) ISO/IEC, “Systems engineering - System life cycle processes", ISO/IEC 15288, 2002. 7) EIA, "EIA-632, Processes for Engineering a System", EIA Standard, Jan. 1999. 8) IEEE, IEEE Standards for Application and Management of the Systems Engineering Process, Dec, 1998. 9) EIA, EIA/IS 731 Systems Engineering Capability Model (SECM), 1994.

ABOUT THE AUTHOR

Yo Chul Choi has worked as a chairman of the company named SEKOREA which offers the Systems Engineering and Project Management services. His main area of interest is in systems engineering consulting and education with provision to specialists in Military and Railway domain. The author has held quite good experience and the actual education record. In particular, the author has carried out various tasks related to user requirement analysis, system modeling,

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 13 of 13 architecting, and test and evaluation in Military, Aerospace, and Railway domain. Moreover the author has played a crucial role in the planning and concept study stage for the project reported in this paper.

Jae Chon Lee, who is a professor of Systems Engineering at Ajou University, has made continuous effort in both education and research to propagate the knowledge of the systems engineering in Korea. Especially he has carried out a lot of projects in Military and Railway systems domain recently. His research interest includes the subjects of modelling and simulation, and model-based systems engineering.

Proceedings of the 3rd Asia‐Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 14

A Systems Approach for Managing Concurrent Product Development Projects

Jun Lina, Tsan Sheng Adam Ngb aSchool of Management, Xi’an Jiaotong University, Xi’an 710049, China bDepartment of Industrial & Systems Eng., National Univ. of Singapore, Singapore

Copyright © 2009 by J. Lin and T.S. Ng. Published and used by APCOSE with permission.

Abstract. In this paper we propose a systems approach to manage product development projects with concurrent processes. Through careful modeling of the rework cycle and the associated feedback loops it is possible to analyze and hence improve the dynamics of the product development project life cycle. To this end we develop a system dynamics model for concurrent product development projects. The ability to model product development processes during the project definition phases provides the project management team with a powerful decision making tool for project strategy formulation. The improvements in project outcome that can be achieved at this stage of the project are potentially massive because of the low costs of change. We illustrate this approach with an application of a mobile phone development project. INTRODUCTION As shown in Figure 1, in the traditional paradigm, product development (PD) process is treated as a series of sequential and functional product development stages (Wheelwright and Clark, 1992). Information generated from one function transfers to the next one only after its completion, which is represented by the unidirectional arrows between development stages in Figure 1. Traditional product development process can be easily managed by Critical Path Method (CPM) and Program Evolution and Review Technique (PERT). These models were initially developed to control schedule, and later expanded to manage resources and costs. Rooted in the traditional sequential paradigm of product development, CPM disaggregates the development process into activities which are related through their temporal dependencies. In other words, the constraints are described as relationships between the beginning and completion of activities. Each activity is treated as a monolithic block of work described only by its duration. These models ignore the interactions between development stages, which are essential for concurrent product development process (Rodrigues and Bowers, 1996; Ford and Sterman, 1998).

Prototype Pilot Product Concept C/D Design C/D C/D C/D Testing Production launch

C/D: Checking & Decision

Figure 1. Sequential and functional product development process

Proceedings of the 3rd Asia‐Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 14

Since the early 1990s, demanding markets and short product life cycles in many industries have forced manufacturing firms to develop low-cost and high-quality products at a rapid pace. At the same time, the increasing technical intensity makes product development more complex. In order to deal with these issues, product development undergoes new trends, such as cross- functional team deployment and concurrent product development. These new trends have increased the uncertainty and complexity of product development. Researchers and practitioners of systems engineering now view product development as a collection of stages which are performed concurrently or iteratively (Figure 2). Traditional tools created for sequential and functional product development are no longer adequate for modeling iterative PD processes.

S1

S2

S3

Stage

Checking & Decision

Information flow Sn between stages Feedback information

Figure 2. Concurrent product development process

Recently, some analytical and simulation models have been developed in the literature to describe concurrent product development process and analyze the trade-offs among project cycle time, quality, and development cost. Smith and Eppinger (1997a, 1997b) developed several analytic models of sequential and parallel design iterations and addressed the effect of iterations among project phases on project cycle time with the Design Structure Matrix. Krishnan et al. (1997) proposed a framework to determine the optimal number and timing of information transfers. They showed that “upstream information evolution” and “downstream sensitivity” are the two important factors affecting overlapping strategies. Loch and Terwiesch (1998) adapted the concepts of evolution and sensitivity: “upstream information evolution” is defined as the continuous design modification process; “downstream sensitivity” represents the impact of a modification on downstream rework. Based on these concepts, they developed an analytical model and derived the optimal communication strategies for overlapped sequential process.

Proceedings of the 3rd Asia‐Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 14

Roemer et al. (2000) analyzed the time-cost tradeoffs in multistage product development. Chakravarty (2001) studied the trade-offs between the overlapping risk and the project time saved. Some special cases were analyzed to establish useful insights for sequential and overlapped processes. Lin et al. (2009a,b) proposed analytical models to improve project performance by balancing the positive and negative effects of overlapping and functional interaction. Although the results of these efforts are insightful in many respects, these models can only be used to analyze the interaction between two development stages. There is a lack of methods to explicitly model and analyze complex concurrent product development projects.

In this paper, we propose a systems approach to manage product development projects. For concurrent and iterative processes, the interactions between multiple development stages become extremely complex and in general not amenable to analysis using mathematical models. Consequently, in this work a System Dynamics (SD) model is used for the purpose of simulating the dynamic behavior of development processes. Systems dynamics is a well-known modeling tool for studying cause-and-effect phenomena of large-scale complex systems, and has found applications ranging from supply chain management, energy forecasting and epidemiology. Sterman (2004) presented an actual case study of the use of SD in demonstrating the impact of design changes and rework of U.S. Navy carriers on downstream completion times. In our work we develop a SD model to capture the interdependency of the concurrent project development stages. Through careful modeling of the rework cycle and the important feedback loops, it is then possible to reproduce closely the dynamic behavior of actual development projects. The SD model allows users to access the impact of different development strategies, such as overlapping and functional interaction, on project performance and dynamics. Therefore, it can help management to identify appropriate development policies.

In order to understand how system dynamics can be applied to manage concurrent product development projects, we first need to understand the rework cycles involved in product development. THE REWORK CYCLE The majority of product development projects fail to deliver their planned objectives, substantially overrunning both cost and schedule targets. Although project planners often consider resource buffers, it frequently fails to cover a realistic amount of rework. This is because we usually can not accurately estimate the effect of development errors. As we know few development errors can be detected immediately. Development errors can linger undetected in supposedly completed tasks for months, or even years. In the meantime, downstream tasks are done on these development errors. In the end, we not only need to correct the development errors but also need to redo the downstream tasks. Such a rework discovery delay can therefore substantially increase the cost of rectifying errors, and may delay project completion.

The concept of the rework cycle is well known and is described in Cooper 1980, Cooper 1993a, and Cooper 1993b. Different from their models, we describe and simulate the rework process in the form of Rework Due to Development Errors and Rework Due to Corruption (Lin et al., 2008).

Proceedings of the 3rd Asia‐Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 14 Rework due to development errors Product development, even for derivative products, is a process with much uncertainty. Consequently, many tasks are incorrectly done in the completion and rework processes. These tasks are termed as Development Errors (DEs). Rework Due to Development Errors refers to rework or rectification of DEs which are identified through review and testing activities. We illustrate the rework process using a stock and flow structure (Figure 3). Stocks represent the accumulation of tasks and flows represent the rates of development activities (Sterman, 2004). Tasks initially reside in the Tasks Remaining (Tr) stock. As the project begins and progresses, tasks correctly done flow into the Tasks-done-correctly (Tc) stock while tasks containing errors or defects add to the Development Errors stock. Development Errors may be identified by a testing activity and flow into the Tasks to be Reworked stock. Therefore the total number of Development Errors decrease as some of them are correctly reworked. Because rework quality is usually not perfect, tasks which are incorrectly reworked flow back into the Development Errors stock. Some of the reworked tasks in the Development Errors stock may need to flow into this rework cycle one or more times. When rework quality is low, this vicious rework cycle dominates the development process. According to Cooper (1993a), “A quality of 0.20 will require five cycles of work and cost (four full rework cycles) to ‘get it right’”.

Tasks-done-correctl y

Redo Tas ks Cor r ec t l y

Compl et e Tasks Cor r ec t l y Tasks to be Reworked

Di s c ov e r Redo Tasks Wrongly De v e l opment Er r or s

Compl et e Tasks Tasks Remai ni ng Wrongly De v e l o pment Errors

Figure 3. Stock-flow Model of Rework Due to Development Errors

Rework due to corruption Rework Due to Corruption refers to rework or rectification when the change of tasks in an upstream phase corrupts the relevant tasks in the downstream phases, whether the downstream

Proceedings of the 3rd Asia‐Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 14

tasks are done correctly or not. In other words, some tasks need to be reworked because they start on incorrect information from upstream phases. We termed this phenomenon as Corruption.

Tasks corrupted are dependent on the reworked tasks of the upstream phase, the Dependency of the development phases, and the progress of the downstream phase (Tasks Done). The reworked tasks of the upstream phase are positively related to Rework Due to Corruption. More changed tasks inevitably mean that more tasks may be reworked in the downstream phases (Krishnan et al., 1997). Dependency represents the relationship between the tasks corrupted in the downstream phase and the fraction of the tasks changed in the upstream phase. It is also positively related to Rework Due to Corruption. Tasks Done accounts for the reason why more rework is needed in concurrent PD process than the rework in sequential process. For traditional sequential product development process most of the Development Errors can be found and resolved before the downstream activities start (at that time Tasks Done of downstream phase is equal to zero and no Corruption arise). For example, in a fully sequential process, pilot production only starts after detail design has been completed and most of the quality problems have been resolved. However, in practice, pilot production usually starts before the upstream activities have been completed in order to reduce project cycle time. Therefore, in today’s concurrent PD process, Corruption accounts for a large portion of rework and affects product development performance seriously (Krishnan et al., 1997).

The stock and flow structure of Rework Due to Corruption is shown in Figure 4. Certain percentage of downstream tasks is completed based on wrong information from Development Errors of upstream phase. These tasks, together with other tasks, reside in the Tasks-done- correctly stock and the Development Errors stock. Corruption occurs when DEs of an upstream phase are identified. The tasks associated with DEs of upstream phases leave the Tasks-done- correctly stock and the Development Errors stock, and then flow into the Tasks to be Reworked stock. Tasks-done-correctl y

Corrupt Redo Tasks Correctl y Tasks-done-correctl y

Compl et e Tasks Correctl y Tasks to be Reworked

Cor r upt Devel opment Redo Tasks Wrongly Er r or s

Compl et e Tasks Tasks Remai ni ng Wrongly De v e l o pment Errors

Figure 4. Stock-Flow Model of Rework Due to Corruption

Proceedings of the 3rd Asia‐Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 14 MODELLING PRODUT DEVELOPMENT PROCESS

Project Articulation Before analyzing a product development project, it is necessary to evaluate the project's scope. For a model to be useful, it must address a specific problem rather than attempt to mirror an entire system in detail. The project scope can normally be determined through analysis of the project budgets and schedules. It is also necessary to capture the management policies and constraints that will drive the project behavior. Examples of these will include the management team’s relative priorities between cost and schedule, and resourcing strategies (overtime versus hiring).

Information Dependencies between Development Stages Before a detailed system dynamics model can be built, we need to have comprehensive information on the dependencies between the various development stages, which can be addressed using a Design Structure Matrix (DSM;Smith and Eppinger, 1997a; Eppinger et al., 1994; Steward, 1981). The DSM method is based on the earlier work in large-scale system decomposition (Ledet and Himmelblau, 1970; Sargent and Westerberg, 1964). An example of its application for mobile phone development is shown in Figure 5. The DSM tabulates information dependencies in a square matrix with the full set of development activities as both row and column labels. Activity names are listed row-wise on the left of the matrix. A mark in an off- diagonal cell represents an information transfer between two development activities/stages. For each activity, its row represents its input and its column shows its output. When activities are listed in temporal order, sub-diagonal marks represent an input from upstream activities/stages to downstream activities/stages. Super-diagonal marks denote a feedback from downstream activities to upstream activities.

1 2 3 4 5 6 7 8 9 10 Completion and rework activity 1 × × × × × of concept development

RC1 2 ×

RC2 3 × Completion and rework activity 4 × × × × × × of detail design

RD1 5 ×

TD1 6 ×

TD2 7 × Completion and rework activity 8 × × × of pilot production

RP1 9 ×

TP1 10 ×

Figure 5. Information flows in the mobile phone development

Proceedings of the 3rd Asia‐Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 14 Dynamic Development Process Model

Tasks-done-correctl y eq_2

Rework Quality Corrupt Tasks-done-correctl y Aver age Rewor k Rat e eq_14

Redo Tasks Correctl y eq_17 Aver age Compl et i on Rat e Rewor k Rat e De pe nde nc e eq_16 Tasks to be Reworked Compl et e Tasks eq_4 Cor r ec t l y eq_8

Compl et i on Discover & Corrupt Rat e De v e l o p ment Errors eq_7 eq_13 & eq_15

Completion Quality

Redo Tasks Wrongly Re wor k Rat e of eq_18 Ups t r e a m Phas e

Compl et e Tasks Tasks Done i n Tasks Remai ni ng Wrongly De v e l o p ment Errors Ups t r e a m Phas e eq_1 eq_9 eq_3

Testing Quality Tas ks Done eq_11 Pr ecedence of Downst r eam Test i ng Rat e of Cons t r ai nt s f or Phas e Do wnst r eam Phas e Compl et i on

Pr ecedence Di s c ov e r y Ra t e Const r ai nt s f or eq_12 Tes t i ng

Aver age Test i ng Rat e Tes t i ng Qual i t y

Tes t i ng Rat e Tes t i ng Remai ni ng eq_10 Tes t i ng Compl et ed eq_5 eq_6

Figure 6. Dynamic development process model (DDPM)

Proceedings of the 3rd Asia‐Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 14

Four stocks and six flows are used to represent the development process of a PD stage. The stocks (Tasks Remaining (Tr), Tasks-done-correctly (Tc), Development Errors (DEs), and Tasks to be Reworked (Ttr)) represent the sizes of the backlogs of tasks. The sizes of the stocks change due to the flows related to development activities. In Figure 6, Complete Tasks Correctly (cc) and Complete Tasks Wrongly (cw) represent the completion activity; Testing Rate (gre) and Discover Development Errors (de) represent the testing activity; Redo Tasks Correctly (rc) and Redo Tasks Wrongly (rw) represent the rework activity; Corrupt Tasks-done-correctly (kc) and Corrupt Development Errors (ke) represent the Corruption caused by upstream rework.

Testing process is represented by two stocks and one flow (Figure 6). Tested tasks leave the Testing Remaining (Gr) stock, pass through the Testing Rate (gre), and then accumulate in the Testing Completed (Gc) stock.

Having built the feedback loops and rework cycles in each development stage, it is possible to inter-link a number of these subsystems using the DSM structure to represent the dynamics of the development phases that constitute an entire project.

Data Collection The input data of the system dynamics model can be determined on historical records, such as project schedule and the quality problems found and solved over the entire period of the project. These data can be double checked together with the engineers familiar with the project. The Average Completion Rate, Average Testing Rate, Completion Quality, and Precedence Constraints can be directly accessed from the historical data (Ford and Sterman, 1998; Black and Repenning, 2001). The other parameters are calculated using following equations: Average Rework Rate is the result of Tasks Reworked divided by Rework Duration; Rework Quality equals Tasks Correctly Reworked divided by Tasks Reworked; Testing Quality equals Development Errors Found divided by Development Errors Exist (Cooper, 1993a); Dependency equals Tasks Corrupted divided by the product of Upstream Tasks Reworked and Tasks Done.

Model Testing Behavior-reproduction tests (Sterman, 2004) can be used to validate the model by comparing simulation results with field data (Figure 7). Many tools are available to assess a model’s ability to reproduce the behavior of a system. Most common are descriptive statistics to assess the point- by-point fit. Point-by-point metrics compute some measures of the error between a data series and the model output at every point for which data exist and then report the average values.

The most widely reported measure of fit is R2, the coefficient of determination. R2 measures the fraction of the variance in the data “explained” by the model. If the model exactly replicates the actual series, R2=1; if the model output is constant, R2=0. R2 is the square of the correlation coefficient which measures the degree to which two series covary.

The mean absolute error, MAE; mean absolute error as a percentage of the mean, MAE/Mean; and root mean square error, RMSE all provide measures of the average error between the simulated and actual series. MAE weights all errors linearly; RMSE weights large errors much more heavily than small ones. Both measure the error in the same units as the

Proceedings of the 3rd Asia‐Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 14

variable itself. MAE/Mean and RMSE/Mean provide dimensionless metrics for the error, which are easier to interpret.

The Theil inequality statistics (Sterman, 1984; Theil, 1966) decompose the mean square error (MSE) into three components: Bias, Unequal Variation, and Unequal Covariation. Bias arises when the model output and data have different means. Unequal variation indicates that the variances of the two series differ. Unequal covariation means the model and data are imperfectly correlated, that is, they differ point by point. Large MAE/RMSE and large bias indicate systematic error and should lead to questions about the assumption of the model.

Figure 7. Reference mode and simulation results

STRATEGY ANALYSIS The developed system dynamics model allows decision-makers and stakeholders to simulate and compare the impact of various project strategies on outcomes of interest. Strategy optimization is then used to seek the desired project outcome. Of course, desirability often depends upon several conflicting requirements of the project stakeholdersFor example a most common tradeoff considered is between project costs and completion schedule.. Going further, the project management team may seek to improve the overall project outcome – i.e. reducing both cost and schedule. This is usually achieved through a combination of changing / improving business processes. Some examples of how potential alternative project strategies alter the project outcome and cost-schedule trade-off are discussed below.

Overlapping of Development Stages Overlapping refers to the product development process where the downstream stage starts prior to the completion of the upstream stage. The primary purpose of adopting the overlapping approach is cycle time reduction through planning and executing multiple stages simultaneously instead of sequentially as in a sequential development process. This requires starting downstream stage as soon as preliminary information is available. For the overlapped process, the development stages are usually sequentially dependent or interdependent. Information generated

Proceedings of the 3rd Asia‐Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 14

by one or more stages poses contingencies for others; thus, all the development stages should be considered simultaneously (Adler, 1995).

Although large reduction in cycle time can be realized by applying overlapping approach (Wheelwright and Clark, 1992; Womack et al., 1990; Nevins and Whitney, 1989), the cycle time reduction comes at the cost of increased complexity. Overlapping increases the dependency between development stages and the number of required information transfers. To deal with the increased interdependencies, intensive coordination is required. However, this may increase the cost of manpower. Because downstream is started on preliminary information in the overlapped process, the amount of rework is likely to increase when new information becomes available.

Cross-functional Teams In today’s product development, functional participation takes place through the formation of teams consisting of representatives from the functions involved. Due to uncertainty in product development processes, the release of preliminary information to downstream functions may introduce the need for rework when there is a change in preliminary information. The goal of functional interaction is to reduce project uncertainty by identifying the potential quality problems as early as possible. The formation of cross-functional teams is an extension of the move away from function-based teams to the matrix structures. Hayes et al. (1988) describe and Wheelwright and Clark (1992) later refine a detailed model of this shift by introducing intermediate steps defined by the level of influence of project managers. Restructuring product development organizations away from function-based groups and toward cross-functional development teams has become a widely used approach to reduce project cycle time (Clark and Fujimoto, 1991).

However, researchers (Clark and Fujimoto, 1991; Dean and Susman, 1991; Takeuchi and Nonaka, 1991) have realized that the formation of cross-functional teams alone does not necessarily reduce time-to-market. They found that over-extended communication and coordination in cross-functional team may lower project performance. Dean and Susman (1991) found that friction between the members from different functions may affect the efficiency of product development. Nevin et al. (1991) listed some other reasons for the cross-functional team failures. Empirical studies show that functional interaction may increase (Eisenhardt and Tabrizi, 1995; Von Corswant and Tunälv, 2002), decrease (Bhuiyan et al., 2004; Wagner and Hoegl, 2006), or have no significant effect (Datar et al., 1997) on project performance. These mixed results indicate that cross-functional team is not a panacea for managing NPD projects. The functional interaction policy should be adjusted according to project characteristics. Thus potential risks must be carefully examined to ensure that added time and effort are kept to a minimum (Krishnan et al., 1997).

Communication Policies Facilitating communication among business functions and/or members in cross-functional teams are commonly used by many companies (Cooper, 1994; Swink et al., 1996; Minderhoud & Fraser, 2005). It is well known that communication among development teams can reduce project uncertainties, but at the expense of additional time and cost for communication. Patrashkova-Volzdoska et al. (2003) reported that communication frequency and performance are nonlinearly dependent with an inverted-U relationship, based on a survey of 60 cross-

Proceedings of the 3rd Asia‐Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 14

functional teams. Helms (2004) observed that the information exchange among development teams is time consuming in chemical industries. In general, if information exchange is too frequent, then communication time and cost would increase significantly. However, infrequent information exchange would delay the identification of the design flaws and increase the amount of rework of the upstream stage. Using the SD model, we can determine the optimal communication frequency that minimizes the expected project completion time.

Sensitivity analysis In practice, it is impossible to accurately estimate all input data and thus sensitivity analysis becomes important. A useful test that we can perform using the SD model is to examine how tolerant the project is to the change of parameter values. Figure 8 shows an example. In practice there are dozens of such sensitivity analyses to perform - iterating each budget, schedule, strategy, etc independently. This permits rapid identification of the key levers for improving project outcome. Obviously, it is desirable to have a baseline project strategy that is able to absorb significant degrees of change with minimum impact.

Figure 8. The effect of opportunity cost of time on overlapping strategy

CONCLUSION Successful new product development is critical for the survival of many companies. The product development literature often provides open-loop, single-link linear relationships which are usually lists of simplified rules for managing projects. These solutions are often not applicable in practice because each firm has a unique set of constraints and requirements, and does not usually carry out the same project twice (Smith and Reinertsen, 1998). For example, in the literature of concurrent engineering, some researchers (e.g. Imai et al., 1985; Takeuchi and Nonaka, 1991; Clark and Fujimoto, 1991; Wheelwright and Clark, 1992) propose that product development processes can be accelerated by concurrent execution of development stages. The others (e.g. Eisenhardt and Tabrizi, 1995; Terwiesch and Loch, 1999) argue that concurrent engineering is not applicable to all product development projects. Although these studies have contributed to qualitative understanding of the problem, more work needs to be developed in order to provide more detailed and implementable guidelines in practice.

Proceedings of the 3rd Asia‐Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 12 of 14

In this work we have provided a systems approach for managing concurrent product development projects in view of the inherent system complexities. The vast majority of product development projects overrun their cost and schedule targets because of rework. Many project management teams are still building future rework into their projects because of their inability to recognize and manage uncertainty in the system design and project delivery mechanisms. By properly using our model we can manage these sources of uncertainty especially during the start- up phase and thus rework can be considerably reduced. Applications of this approach include: cost-schedule-quality trade-offs, developing project improvement strategies, risk mitigation planning, and project robustness testing. A properly optimized project strategy can significantly reduce rework, and lead to a massive improvement in project outcome. REFERENCES Adler, P.S., 1995. Interdepartmental interdependence and coordination: the case of the design / manufacturing interface. Organization Science 6 (2), 147-167. Bhuiyan, N., Gerwin, D., Thomson, V., 2004. Simulation of the new product development process for performance improvement. Management Science 50 (12), 1690-1703. Chakravarty, A.K., 2001. Overlapping design and build cycles in product development. European Journal of Operational Research 134 (2), 392-424. Clark, K.B., Fujimoto, T., 1991. Product Development Performance Strategy, Organization and Management in the World Auto Industry. Harvard Business School Press, Boston, MA. Cooper, K.G., 1980. Naval ship production: A claim settled and a framework built. Interface 10 (6), 20-36. Cooper, K.G., 1993a. The rework cycle: Benchmarks for the project manager. Project Management Journal 24 (1), 17-22. Cooper, K.G., 1993b. The rework cycle: How projects are mismanaged. PMNETwork, February, 5-7. Cooper, R.G., 1994. Third generation of new product processes. Journal of Production Innovation Management 11 (1), 3-14. Datar, S., Jordan, C., Kekre, S., Rajiv, S., Srinivasan, K., 1997. New product development structures and time-to-market. Management Science 43 (4), 452-464. Dean, J.W.Jr., Susman, G.I., 1991. Organizing for manufacturable design: managing product life cycles from start to finish. Harvard Business Review Paperback. Cambridge, MA. Eisenhardt, K.M., Tabrizi, B.N., 1995. Accelerating adaptive processes: Product innovation in the global computer industry. Administrative Science Quarterly 40 (1), 84-110. Eppinger, S.D., Whitney, D.E., Smith, R.P., Gebala, D.A., 1994. A model-based method for organizing tasks in product development. Research in Engineering Design 6 (1), 1-13. Ford, D.N., Sterman, J.D., 1998. Dynamic modeling of product development processes. System Dynamics Review 14 (1), 31-68. Hayes, R.H., Wheelwright, S.C., Clark K.B., 1988. Dynamic Manufacturing, Creating the Learning Organization. New York: The Free Press. Helms, R., 2004. Framework for releasing preliminary information in product development. Advanced Engineering Informatics 18 (4), 231-240. Imai, K., Nonaka, I., Takeuchi, H., 1985. Managing the new product development process: how the Japanese companies learn and unlearn in The Uneasy Alliance. Clark, K.B., Hayes, R.H., Lorenz, C., eds. Harvard Business School Press, Cambridge, MA.

Proceedings of the 3rd Asia‐Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 13 of 14

Krishnan, V., Eppinger, S.D., Whitney, D.E., 1997. A model-based framework to overlap product development activities. Management Science 43 (4), 437-451. Ledet, W., Himmelblau, D., 1970. Decomposition procedures for the solving of large scale systems, Advances in Chemical Engineering 8, 185-224. Lin, J., Chai, K.H., Brombacher, A.C., Wong, Y.S., 2009a. Optimal overlapping and functional interaction in product development. European Journal of Operational Research, 196 (3), 1158-1169. Lin, J., Qian, Y., Cui, W., Miao, Z., 2009b. Overlapping and Communication Policies in Product Development, European Journal of Operational Research, forthcoming. doi: 10.1016/j.ejor.2009.03.040 Lin, J., Chai, K.H., Wong, Y.S., Brombacher, A.C., 2008. A dynamic model for managing overlapped iterative product development. European Journal of Operational Research 185 (1), 378-392. Loch, C.H., Terwiesch, C., 1998. Communication and uncertainty in concurrent engineering. Management Science 44 (8), 1032-1048. Minderhoud, S., Fraser, P., 2005. Shifting paradigms of product development in fast and dynamic markets. Reliability Engineering and System Safety 88 (2), 127-135. Nevins, J.L., Whitney, D., 1989. Concurrent Design of Products & Processes, A Strategy for the Next Generation in Manufacturing. McGraw-Hill, New York. Nevins, T.M., Summe, G.L., Uttal, B., 1991. Commercializing technology: what the best companies do: managing product life cycles from start to finish. Harvard Business Review Paperback. Cambridge, MA. Patrashkova-Volzdoska, R., McComb, S.A., Green, S.G., Compton, W.D., 2003. Examining a curvilinear relationship between communication frequency and team performance in cross- functional project teams. IEEE Transactions on Engineering Management 50 (3), 262-269. Rodrigues, A., Bowers., J., 1996. System dynamics in project management: a comparative analysis with traditional methods. System Dynamics Review 12 (2), 121-139. Roemer, T.A., Ahmadi, R., Wang, R.H., 2000. Time-cost tradeoffs in overlapped product development. Operations Research 48 (6), 858-865. Sargent, W., Westerberg, A., 1964. Speed-up in chemical engineering design. Transactions of the Institution of Chemical Engineers and the Chemical Engineer 42, 190-197. Smith, P.G., Reinertsen, D.G., 1998. Developing Products in Half the Time, 2nd ed. Van Nostrand Reinhold, New York. Smith, R.P., Eppinger, S.D., 1997a. Identifying controlling features of engineering design iteration. Management Science 43 (3), 276-293. Smith, R.P., Eppinger, S.D., 1997b. A predictive model of sequential iteration in engineering design. Management Science 43 (8), 1104-1120. Sterman, J.D., 1984. Appropriate summary statistics for evaluating the historical fit of system dynamics models. Dynamica 10 (2), 51-66. Sterman, J.D., 2004. Business Dynamics: Systems Thinking and Modeling for a Complex World. Irwin/McGraw-Hill, Boston. Steward, D.V., 1981. The design structure system: A method for managing the design of complex systems. IEEE Transactions on Engineering Management 49 (4), 428-442. Swink, M.L., Sandvig, C., Mabert, V.A., 1996. Customizing concurrent engineering processes: five case studies. Journal of Production Innovation Management 13 (3), 229-244.

Proceedings of the 3rd Asia‐Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 14 of 14

Takeuchi, H., Nonaka, I., 1991. The new product development game: managing product life cycles from start to finish. Harvard Business Review Paperback. Cambridge, MA. Terwiesch, C., Loch, C.H., 1999. Measuring the effectiveness of overlapping development activities. Management Science 45 (4), 455-465. Theil, H., 1966. Applied Economic Forecasting. North Holland Publishing Company, Amsterdam. Von Corswant, F., Tunälv, C., 2002. Coordinating customers and proactive suppliers—a case study of supplier collaboration in product development. Journal of Engineering and Technology Management 19 (3–4), 249−261. Wagner, S.M., Hoegl, M., 2006. Involving suppliers in product development: insights from R&D directors and project managers. Industrial Marketing Management 35 (8), 936-943. Wheelwright, S.C., Clark, K.B., 1992. Revolutionizing Product Development. The Free Press, New York. Womack, J.P., Jones, D.T., Roos, D., 1990. The Machine that Changed the World, The Story of Lean Production. Rawson Associates, New York. Biographies Jun Lin is currently an assistant professor in the School of Management, Xi’an Jiaotong University. He was previously a research fellow in the Department of Industrial and Systems Engineering, National University of Singapore. He received his doctorate from National University of Singapore and Eindhoven University of Technology in 2008 in the field of Engineering Management. His research interests include system dynamics, modeling of product development process, production planning. He has co-authored several journal papers on the topic of system dynamics and project management. His e-mail address is [email protected].

Tsan Sheng Adam Ng is an assistant professor with the Department of Industrial and Systems Engineering, National University of Singapore. His research interests include operations research applications in production planning, manufacturing supply chain, and large scale manpower modeling using system dynamics simulation. His email address is [email protected].

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 18

System Engineering’s Role in Today’s Mega Projects - “The Systems Integrator” -

Leon “Lee” J. Pandazides, BS-EE, MS-Q+R Principle, LJP & Associates - RESULTANTS Advisors to Management for Enhanced Systems for Managing Asian Offices – Taichung, Taiwan [email protected]

Copyright © 2009 by Leon J. Pandazides. Published and used by APCOSE with permission

Abstract. Major Infrastructure Projects now exceed USD $1Billion+ (Mega Project Status). Public Private Partnerships (PPP) are being formed with Concessions/Consortiums to Design, Build, Operate/Maintain, and even required to fund these Projects through Private Financing Initiatives (PFI), backed by Stakeholder equity from Industrial and Institutional Partners; Multi-national Concessions; and with Stiff “Commercial Penalties” if Performance Availability Guarantees are not achieved over extended periods of time (25+years).

The challenges for System Engineering is to look at the entire Project Life Cycle from Concept to Decommissioning, to ensure cost considerations are made to achieve and maintain long term Performance and Availability Guarantees. System Engineering, Project Management & Assurance, Commercial Management, Suppliers, and Operations & Maintenance, must integrate to create life cycle cost saving initiatives. Times have changed!!

This Paper tells why System Engineers must transform into “Systems Integrators”; Shares a Case Study - A Mega-Project; Talks about integrating management systems on Projects; and offers Tips on Things-That-Worked.

Introduction

Most major infrastructure projects are created by the Public Sector to provide needed improvements for their communities. Typically, these new infrastructure projects take from 10 to 20 years just to go from Concept to Implementation. Funding for such projects historically was secured through federal, regional and/or local agencies. With more and more competition for less and less money, the public sector is turning to the private sector for help. Public-Private-Partnerships, “PPP” have evolved, whereby, public and private sectors work in partnerships to implement Mega-Projects. Concession Consortiums are funding these projects through Private Financing Initiatives (PFI), backed by Stakeholder equity from Industrial and Institutional Partners; Stiff “Commercial Penalties” if Performance Availability Guarantees are not achieved; and Multi-national Concessions. Lenders and Lawyers have now moved into the forefront when it comes to Mega-Projects. In many cases, the Lenders and Lawyers are to “co-pilots” during developments of Mega- Projects, thereby placing higher focus and emphasis on commercial risks. The essence of these partnerships is to find a balance for the allocation of risk. While this Paper does not specifically focus on Risk Management, in today’s environment where funding is limited, Project Risks must be the focus for overall management of Projects. What follows is a case study of the development for Holland High Speed Rail Link (HSL) between Amsterdam and Brussels. Some terms used in this Paper, such as “Phases & Phrases” of a Project are as defined in the European Standard, EN-50126, The Standard for Railway applications - The specification and demonstration of Reliability, Availability, Maintainability and Safety (RAMS). Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 18

Case Study – Holland’s High Speed Line-South Project – HSL-Zuid

Planning for the High Speed Line-South (HSL-Zuid) Transport System began in the 80’s when the European Union (EU) laid out it’s plans for a high speed rail network to connect the major cities of Europe. The successes and public acceptance of high speed rail transportation implemented in France, Germany, Italy and Spain encouraged this initiative. Brussels, as seat for the EU, was designated as a major hub for high speed rail. Holland, bordering Belgium, needed to join the network with a high speed link (HSL) joining Amsterdam to Brussels, through Rotterdam. The network for the Holland HSL and the key technical data for the System is shown below in Figure 1.

Key Technical Data • The Network: Two tracks capable of bi-directional operation at full line speed. - right hand running north of Rotterdam to AMS (43 km “Northern Line Section”) - left hand running south of Rotterdam to border (45 km “Southern Line Section) • Train speed: - International trains 300 kph - Domestic trains 220 kph • Optimal operational headway: 180 sec • The tunnels: 4 – for a total of 17.9 km incl. Ramps, 1 Shield driven • The (major) bridges / viaducts: 2 – Viaducts; 1 – Bridge; 1 Aqueduct • Curves: 1.600 meter 160 kph 4.000 meter 300 kph • Grades max. 2,5%

Figure 1. Holland HSL and the key technical data for the System

The HSL-Zuid: PPP Model – Clear roles for inter-relationships and balanced allocation of Risk: Figure 2 below shows the Holland Private Public Partnering (PPP) Model. It provided a clear role for each of the four entities contributing to the overall High Speed Line- South Transport System. The State works with their various Regulators making sure measures are taken on the Public side to support the Private side, while the Private sector works with the Public side to support Public sector needs.

PRIVATE PUBLIC

TrainTrain OperatingOperating ComCom pany pany GovernmentGovernment International/DInternational/D om om estic estic HSAHSA (High(High SpeedSpeed AllianceAlliance -- NS/KLM NS/KLM) )

RegulatorRegulator RVLRVL NS NS ControlControl CentersCenters TTim im e etabling ta b lin g T Traffic raffic CC o ontrol n tro l „„ AuditAudit PP ow ow er er CC ontrol ontrol „„ ApprovalApproval „„ RegulatingRegulating InfraproviderInfraprovider ConcessionaireConcessionaire forfor SystemSystems, s, „„ StateState InsurancesInsurances TrackwTrackw ork, ork, Signalling,TelecomSignalling,Telecom m m unication, unication, PowerPower Supply,Supply, NoiseNoise Barriers,Barriers, FinancingFinancing (D(D esign esign , , CC on on stru stru ction ction , , MM ain ain ten ten an an c c e) e)

CivilAssetsCivil Assets StateState SixS ix sections,s ection s, one on e shieldsh ield drivendriven tunneltun nel providesprovides financefinance

Figure 2. Holland HSL PPP Model Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 18

The contracts for the six civil sections and shield driven tunnel were let first so the tunnel, bridge, viaduct, and aqueduct foundations could be prepared in advance for the superstructure to be provided by the Infrastructure Provider (IP). Tendering for the IP started in early 1999 with the qualification of four Consortiums to bid for the job. The High Speed Line-South (HSL-Zuid) brought some firsts to the rail industry. • In 1998, the EU issued a directive requiring independent certification of equipment and systems to prove cross border interoperability of high speed rail service. The HSL was to be the first project to operate across it’s border with Belgium at high speed. France, Germany, Italy, and Spain all operated within their national borders. • To better ensure the availability performance guarantees, the State required the new standard: CENELEC EN 50126, Railway Applications – The specification and demonstration of Reliability, Availability, Maintainability, and Safety (RAMS), become the standard to be followed for project realization. The HSL was to be the first project to be realized completely under this standard. See below for a short discussion describing the EN 50126 standard. • ISO-9001 was to be the model for the project quality management system, however, no certification was required. ( However, Certification was done) • EN 50126, brought with it, and the State demanded, a robust safety management system be put in place, to be fully documented in a series of Project Safety Cases/Reports, and assessments by an Independent Safety Assessor. • Verification and Validation of specific technical Requirements of the contract must demonstrate compliance under Requirement Management and Requirements Tracking before the State issues the Certificate of Availability. The HSL is the first to require such Requirements management and tracking process for all elements of the project. Before the final Implementation Agreement and Financial closure between the State and their Infrastructure Provider could be signed, 133 individual contracts were required to be executed. Infraspeed won the contract to be the Infrastructure Provider for HSL. Figures 3a and 3b below, provides a contractual overview for the project, and Shareholder equity - Partner scope of work.

CONTRACTUAL OVERVIEW INFRASPEED EP C Consortium VO F CLIENT Dutch Government (MoT/MoF) Siemens, BAM, Fluor Infrastructure HSL ZUID Org. (IPCM) Concession Contract fixed price 133 date cer tain Obligation to pay Performance Fee lump sum turnkey LEND ERS contract Individual Rabobank ING Bank Contracts KBC Bank KfW Dexia Credit Local Provision of Debt HypoVereinsbank Special Purpose Vehicle Payment of Principal & Interest Various Syndicated Banks Management Services Flu or Infr as tructure BV INFRASPEED BV European Investment Bank

LEND ERS’ ADVISORS Allen & Overy (Legal) INFRASPEED Maintenance BV Index linked 25 yrs Maintenance Contract, Scott Wilson Railways (Techn) incl.renewals 100% Shareholding 100% Shareholding Heath Lambert (Insurance) INFRASPEED Hol dings BV PKF (financial auditor)

Siemens, BAM, Fluor Insurance Providers Infrastructure Sh areh o lder Fun ds INFRASPEED ADVISORS Freshfields Bruckhaus Derin ger (Legal)

Industrial Investors Institutional Investors Ernst & Young (Accounting, Fiscal) Siemens, BAM, Fluor Innisfree, Infrastructure HSBC In frastructure M arsh (Insurance) Railcert (Independent Safety Advisor)

Figure 3a. Contractual Overview of the Holland HSL Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 18

Shareholder Equity - % Partner Scope of Work before and after BAM Group ……………………………………….42% conversion (‘07) • Slab Track BAM Group 21.5 10.5 • Noise barriers and fencing Fluor Infrastructure 7.1 3.5 • Misc Buildings and yards Siemens 22.4 11.0 Siemens ……………………………………………44% Industrial Partners 51% ---> 25% • Systems design – Systems Integration Signaling System Innisfree 24.5 37.5 • Communications System HSBC Infrastructure 24.5 37.5 • Traction Power Supply and Distribution Institutional Partners 49% ---> 75% • All Ancillary Equipment Fluor … Program Management……………………11% Bid and Development Cost __ 3% Total Construction Costs ……… ……………….100% Figure 3b. Shareholder Equity and Partner Scope of Work

Essential Features: The PPP contract to the Infrastructure Provider, Infraspeed, was the largest PPP Contract ever awarded by the Dutch Government to a Private Party. It involves: Design, Build, Finance, and Maintenance of the superstructure of new 100 km high speed rail link between Amsterdam and Belgium border. Initial Funding Requirement of € 1.2 billion. Concession term :2001 to 2031; 5 year Design – Build Development Period; 25 year Maintenance Availability Period; € 3.0 billion performance based revenue, covers investment cost, maintenance cost, cost of amortization, taxes, insurances, cost of capital, profit and overhead. The preceding was provided in somewhat more detail then a normal project overview to The Holland High Speed Rail Link provided complexities and intricacies typical for today’s Mega Projects. In many ways, all project have similar things to deal with, however, Mega Projects involve complexity above the routine of most Project Practitioners. Figure 4 below, shows the essential features for the Project. Before installation of the subsystems, the Civil Sections passed through a detailed Handover process implemented to assure the “As Received” condition of the job site is clearly established and documented. Site surveys, inspections, and tests were performed to establish the baseline for Handover. Resolution complexities often end up on the Project Risk Register.

Noordelijk Holland Hoofddorp „ Sub Structure as RAS Shield-driven Tunnel “Groene Hart” taken from Civil Assets Provider Zuid Holland „ Slab Track System Midden Rotterdam „ Noise Attenuation West RAS Barendrecht Barrier, Fences, and Rotterdam Misc. Buildings Lombardijen Zuid Holland Zuid „ Signaling „ Communications „ Breda RAS Brabant Traction Power Noord Supply and Brabant Distribution Zuid „ Ancillary

Figure 4. Features of the Holland High Speed Rail Project Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 18

HSL System Architecture: Figure 5 below, provides insight as to the HSL System architecture and the System design. The System architecture breaks the complete system down into characterized subsystems/functions and identifies the internal and external interfaces. The architecture is the basis for the System design which develops a physical design, documents the complete configuration for the manufacturing and installation phase. Technical information which has specific system relevance or which is common for all Sub Systems will be harmonized, integrated into the system design and controlled through all phases. This information is mainly linked to EMC, cabling, layouts, reliability, availability, maintainability, safety, norms and standards, internal and external interfaces, functional breakdowns and the subsystems architectures. The identification of the interfaces is the baseline for the interface management with the purpose that each partner has the right information at the right time. Specific attention is paid to civil interfaces, operational procedures and to all external interconnections and confirmations.

Figure 5, Simplified System Architecture for the HSL Infrastructure Provides Subsystems

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 18

EN 50126 V Model Overview for Verification & Validation (V&V): For the HSL, the Client required to Infrastructure Provider to follow the “V” Model requirement from EN 50126 for Systems Engineering and Verification & Validation. Figure 6 below, shows how the HSL Client Requirements for the Subsystems were received and thereafter Apportioned to the individual Subsystems. The Subsystem Suppliers thereafter Apportioned their Requirements down to the individual modules/equipments that made up the Subsystem. The next Phase was to Design & Develop each Subsystem and confirm thru Design Reviews that Requirements have been incorporated into the design of each Subsystem. Pre- Site Testing at Manufactures facilities confirmed Requirements were incorporated into the manufactured items. During Installation Phases, each piece of equipment had to pass rigorous inspection and testing. After Installation was completed, the Subsystems faced intensive Commissioning and Integration Testing. The Testing was topped of by high speed train runs at 300 km/hr to demonstrate Subsystem capabilities. All documentation from activities performed during each Phase was fed into the Project Electronic Documentation and Archival Management System (EDMS) to provide the documented evidence as proof of compliance to Internal Auditors, Third Party Technical Advisors/Auditors, Notified Bodies, Financial Technical Advisors, Safety Report, Requirement Compliance Matrix, and lastly for the Certificate of Availability signaling end of Development Period and beginning of Availability Period.

HSL Requirements

Requirements Compliance Matrix (RCM) with Agreed V&V

“V” Model (50126) Verification & Validation Process (QMS)

Apportionment Subsystem Validation Analysis, Study, Assessment, Design & Devel Installation Static Testing Commissioning Dynamic Calculation, Integration Testing Evaluation of Similar Solution Manufacture (SACE) Realization Process

Modeling/ Simulations (DSRV) (PRST) (OSIN) (OSTC) (DYNT) Design Reviews Pre-Site Testing On-Site Inspections Static Testing Dynamic Testing(SIM1) (SIM2) (SIM3)(SIM4) (IPRA) Internal Process Audits (SIM5)(SIM6) (XCRT) E x t e r n a l C e r t i f i c a t i o n – N o t i f i e d B o d y – I n t e r o p e r a b i l i t y

Electronic Documentation Management System (EDMS) PLUS + Tracking

Safety Cases – Requirements Compliance Matrix Certificate of Lenders Consents Development/Availability Certificate (RCMC) Conformity(NoBo) TA Approvals

Figure 6, Verification & Validation Model for Compliance

Throughout the 25 year Availability Period, the Subsystems would be continuously monitored to determined if the Subsystems maintained their required 99.6% Availability guarantee. A complex and sophisticated computerized Performance Payment Regime , developed during Development of the Project was operated to report each month the achieved Availability percentage. Ever month the Subsystems achieved their 99.6% guaranteed Availability rate, the Concession was paid their agreed monthly payment. If the Availability figure was not achieved, the payment would be proportionately reduced. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 18

Integrating Management Systems for Projects®(IMSP®)

Integrating Management Systems for Projects® was created and tested on the Holland High Speed Rail Project by the Author. It proved invaluable. What follows is an overview.

Objectives: The primary objectives of IMSP® are to: a) Provide a process driven, risk based, management tool that places maximum emphasis on assuring all project related requirements have been achieved and the objective evidence of their individual compliance is available and stored for electronic archrival and retrieval. b) Monitor each Key Process using Key Process Indicators (KPI) for real time feedback on process performance measurements, thereby allowing quicker response for risk sensitive areas. The higher the risk, the closer the monitoring. c) Align and integrate everything that has an effect on business results as a Key Process and as such is part of the overall management system for the Project. d) Place greater emphasis on System Integration and Assurance, i.e., prevention and measurement; e) Respond to the increasing requirements and demands of industry; f) Provide integrated performance monitoring of Key Process’ for real time Performance monitoring and Risk Assessment. g) Provide the level of seamless transparency being demanded by both financial institutions and Clients for demonstrating proof of compliance. h) Assure that the objective evidence of compliance has been created and safely stored.

The IMSP® Key Processes: Research of past turn-key project indicated 30 Major Processes are typical to every project. Each of the “Key Processes” can have Sub-Processes. Table 1 below, shows what can be generally considered to be the Key Processes for a Project

Table 1. Key Processes for integrating management system for Projects # Management Process # Management Process 1 Project Management 17 Client Training Management 2 Project Organization 18 Project Risk Management 3 Project Controls 19 Project Security Management 4 Project Quality Management 20 Project Asset Management 5 Project Environmental Management 21 Project Acceptance Management 6 Project Documentation and IT Management 22 Project Closeout Process 7 Project Configuration Management 23 Project Handover Management Project Design and System Engineering/ Team Building – Leadership Development; 8 24 Integration Management Strategic Planning - Facilitation 9 Project Interface Management 25 Project Maintenance Management 10 Project RAM Management 26 Project Contract Management 11 Project Safety Management* (*incl. H&S) 27 Project Insurance Management 12 Project System Assurance Management 28 Project Warranty Management 13 Project Procurement Management 29 General Project Office Management 14 Project Construction Management 30 Project Commercial/Financial Management 15 Project Testing and Commissioning 31 Project Administration Management* (*+HR) 16 Project Consents Management 32 Project Operations & Maintenance Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 18

Bases for the IMSP®: Figure 7 below, shows the fundamental breakdown of the IMSP® .

Figure 7. Key Processes for an integrated management system for Projects Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 18

IMSP® Process Model - Circle of Power – SYNERGY: Figure 8 below, illustrates the alignment of the IPSM® processes for integration thru the central core. In Figure 7 below, the Design & Engineering Management process is highlighted showing its direct connection to the core of the overall management system and its “Red Thread” connection with all other process of IMSP®. Figure 11 in the below section “Functional Team Organization”, shows how the functional integration goes directly to Sub- system suppliers and Partners

Figure 8. Process alignment for integration to the Central Core

As part of the robust assurance process’ contained in the IMSP®, Quality, Risk, and System Assurance aspects are more closely monitored as shown in Figure 9 below.

Figure 9. Assurance Monitoring for integration to the Central Core Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 18

IPSM Documentation Model – Evidence of Compliance: Figures 10a and 10b below, show how the IMSP® extends over the Consortium project lifecycle to assure adequate documentation creation and retention for compliance demonstration. ¾ For the Acquisition Phase/Bid Phase, proposals in which Concessions are required to submit information on its proposed project management activities, the concept and approach as to What would/will be done “IF” they were to receive the award. The Bid Team will find bid template Management Plans for each of the major processes of the IMSP® on line for their use. This assures global access and consistency of proposal information. ¾ For the Design and Implementation Phase, Concessions must maintain a, project specific, Project Procedures Manual (PPM), in which is contained o The Management Plans are updated and made specific to the project. The Plans are to define What will be done by the Consortium, when these activities’/tasks will be done, and reference, the supporting procedures, in which it is told, How it will be done. o The Project Procedures are to be made project specific defining How things are done and identify what documentation will be created to be filed to record compliance. o The Implementation Team will find template implement plans and procedures for each of the major processes of the IMSP® on line for their use. This assures global access and consistency of implementation information.

ACQUISITION BID DESIGN & IMPLEMENTATION PHASE ACCEPTANCE PHASE PHASE

Figure 10a. Documentation Model to Demonstrate Compliance

In all cases, the focus is to maintain as much consistency as possible thru use of standardized templates for Project Plans and Procedures. The Plans and Procedures templates are created based on past experience and lessons learned. This helps to better assure the project will be competed on time, within budget and attain the quality defined by the contract and expected by the Client.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 18

Lessons Learned

ACQUISITION BID IMPLEMENTATION ACCEPTANCE PHASE PHASE PHASE

BID DOCUMENTS IMPLEMENTATION DOCUMENTS RECORDS

RECORDS RECORDS Implementation Proof Proof Conceptual Management Plans of of Management Plans Generic - yet – Specific Compliance Compliance Generic - yet – Specific “WHAT will do,and “WHEN”. “WHAT will done, Refer to “HOW”! and “WHEN”. “NEVER TELL HOW” RECORDS RECORDS Proof Proof of of TOC of Proposed Project Specific Compliance Compliance Project Specific Project Procedures Manual Project Procedures Manual (PPM) (PPM) Plans = What + When; Procedures-Instructions = RECORDS HOW! Proof Defines Records to be created of to serve as proof of compliance. Compliance

Figure 10b. Documentation Model to Demonstrate Compliance

IMSP® Functional - Team Organization Approach: Figure 11 below, graphically illustrates the typical “FUNCTIONAL” organization for IMSP® implementation. The layout would be typical for most EPC style projects Consortium is involved. The purpose of the Figure is to show functional/process connections for integration. The Systems Integrator is used as example to show how the Systems Interrogator function/team integrates with each of their respective Subsystem Supplier, Civil Joint Venture type Partner, and other counterparts. The Consortium always Leads the process and nominates an individual in the Consortium organization to act as “Owner” on its behalf. In certain cases and on certain projects, “Ownership” of the IMSP® function/process within the EPC Consortium is contracted out or delegated contractually to other Consortium Partners.

Figure 11. Functional Integration of Key Project Processes Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 12 of 18

Organization for IMSP®: Figure 12 below, illustrates how a Concession would organized for Integrated Project Management System® (IMSP)® implementation responsibility for a typical Engineering, Procurement, and Construction (EPC) Consortium, on which, the Consortium is responsible for overall management of the Project during the Design- Engineering, Procurement, and Construction (EPC) part of the Development Period of a Turnkey Project. The Overall Project Management has a Core Group to provide project management direction, and oversight of the internal controls put in place for the project. The Commercial Management Group is responsible for the Commercial, Financial and Administrative Management controls put in place for the project. The Technical Management Group is responsible for managing the design, engineering, controls, and integration, as well a management of Project Requirements for the Turnkey System. This includes assuring compliance to the contractual requirements for System Acceptance; Operation and Maintenance Concepts and Warranty administration. Subsystem Supplier Project Management Systems are aligned and integrated with the Consortium IMSP® on a direct one- to-one basis to better assure top – down and bottom – up requirements compliance thru out the organization.

Figure 12. Typical Mega Project Organization with IMSP®

During the Design and Implementation Phases of the Development Period, the Subsystem Equipments will be manufactured and transported to the sites for installation on the infrastructure facilities also typically constructed by the Consortiums Civil Joint Venture (CJV), Consortium Partner, responsible for managing construction of the project. Testing and Commissioning will lead to Acceptance of the Project. The Project will then transition over to the Operation and Maintenance Periods. For the (O&M) Periods, the management of the Operations may be performed by an Operating Company or the Client. The same holds true for the management of the Maintenance. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 13 of 18

Areas of Interest – What Worked

Offered below, for consideration, are suggestions, recommendations and tips as to what has worked when trying to manage Mega-Projects or on any project. These are activities/areas typically needing consideration, for they all will have an influence on the Project, System Integration, Quality Management System, and Project Risks. This listing is by no means all inclusive or provided in elaborate detail. There are no guarantees promised that what has worked on one project, could or would work on another. Each Project is different and each has their individual idiosyncrasies. The true challenge for tomorrows System Integrators will be to be able to extract from their tool boxes the essentials to help make the project a success, by Achieving Excellence for your Clients®.

Adding Risk Based - to Process Driven, Management System implementation: System Integration is undergoing transformation. Risk now directly linked to System Engineering. The traditional “SE”, is now the “Systems Integrator”. All Management Systems of a Mega- Project must be Risk Based. Some of the first questions, therefore, to be asked are: • “What Project Requirements pose the highest risk to the Project?”; • “How are we Managing the Verification and Validation of each Project Requirement?” • “What are the processes related to these High Risk Requirements?”; • “What is the mitigation established for each of these Risks”; • “For each Process, work with the Process Leader to clearly define the processes”; • “Each of these Processes are included in the Project Quality Management System”; • “For each Process, define key points for measurement, what is to be measured, when to measure, and What measurement should be expected”; • “Empower each Process Leader with Process responsibility – Input and Output”; • “Identify other processes with which interface is possible/expected”; • “Establish control for all process interfaces”; • “Speak about quality in terms of Risk”; • “Establish clarity in process relationships”; • “Strive for a balanced allocation of Risk”; • “Continually analyze Risk and update related Processes accordingly”;

Interface Management: Managing interfaces is a major risk area for Mega Projects. This is due to the ever increasing interdependencies processes have with each other. While there are many methods for managing project interface, they all must follow a systematic approach to define and resolve all interfaces with each process, function, and/or connection. One of the simplest tools for defining and resolving interfaces is an Interface Control Form. Identify each Interface and establish applicable controls. Track Interfaces to assure resolution. ƒ Management Systems Integration: The “Management System” style and structure created by the ISO-9000 Standards for manage quality, are being adopted for other project processes, such as Safety Management, RAM Management, Configuration Management, Project Management, and others. Integration of ISO-9001 Quality Management Systems on a Project is eased by the global recognition and acceptance of the ISO-9000 movement. Unfortunately, the same cannot be said for the “other” Management Systems, such as Safety Management Systems, Configuration Management Systems, RAM Management System, etc. Today’s Mega-Projects are faced with the dilemma of how to integrate all systems into the overall project, instead of having them each operate independently, and each vying for management attention and priority. The SOX (Sarbanes-Oxley Act 2002) Compliance now requires Commercial/Financial Processes to be integrated as well; Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 14 of 18

Build Team Building, Leadership Development, & Strategic Planning Processes as an integral part of the overall Management System for the Project. With multi-cultural (Nationality and Corporate) Concessions, the need for “Integration” of the Management Teams of the Project is key to the success of the Project. Training to be provided by professional Facilitators. Helps to “Blend” National and Corporate cultures to allow better working Teams. Leadership Development helps expose and enhance leadership skills of Leaders and future Leaders on Project.

Project Plans and Procedures – Hire Technical Writers at Start-Up: For most Projects, but always for Mega Projects, a Project Procedures Manual (PPM) needs to be developed in which the Plans and Procedures specify the various processes of the Project are defined. The PPM becomes the basis for documenting compliance. The Plans define the “Who?, What’s and When’s”, and the Procedures and Instructions describe the “How’s”. Most Engineers are not writers. Proactive Management teams are now seeing the value in having “Technical Writers” brought in for a short period, in the very beginning – Start-up, as the Project office is being set up and established, to sit with the Process Leaders to finalize the Plans and Procedures for the Project in a language style friendly to readers. At the same time key milestones are defined for measurement and what is to be expected at each of the measurements. The language most common to international projects is English. However, there are basically, two styles of English – American English - and - British English. When it comes to writing style for Contracts and Technical documents, American English is the preferred language/style. The American language/style is seen as being more “descriptive” and understandable. British English is seen as more “prescriptive” and complex. Use American English.

Key Process Indicators (KPI) for Improvement: Establishing Key Project/Process Indicators (KPI) is necessary for continuously monitoring high risk project processes and to look for individual process improvements. The best time to establish KPI’s is during the early phases at start-up, when Project execution plans are being formulated. Nominated Process Leaders must plan and describe what the process will do, when it will be done, and how it will be done. KPI’s allows Management to check/monitor the Pulse of the Project. Mega-Projects are dynamic in nature and because they run for extended periods of time, improvements in key processes will bring efficiencies over the long term, especially when it comes to technological improvement. Always ensure Customer Satisfaction and Complaints are Key Project Indicators monitored and with results posted throughout the Project. Iclude them on the “Project Score Card”.

Contractual Change, Variation and Claims Management: While Change Management in the technical sense is known, Contractual Changes/Variation typically belong to our Commercial/Contractual project colleagues. The Management Systems for the Project needs to include these processes also as a management function, in-that they are usually a commercial expression of technical changes/variations. From the management standpoint, the monitoring of the Process would be limited to Management and Products of the Change/Variation process, not the financial aspects of it. Because the Management Principles are based on providing documentation to be used as objective evidence, these same principles are applied to ensure documentation is provided to substantiate Contractual Changes and Variations. In many cases, the documentation often already exists to support our Commercial colleagues quest.

Learn to use Story Boarding: The idea of Story Boarding initially came from the Movie industry. They came up with this simple idea to allow the Director of the movie to clearly see how the “scene” is suppose to play-out or look like, and see what story was to be told, Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 15 of 18

before he got into a costly production. Today’s Directors of Mega Projects need the same kind of visibility. Story Boards can be as simplistic as a long piece of drawing paper/ vellum rolled out as a scroll on a table, to computer graphics able to be reduced to the lowest possible level of detail. Computer enhanced Story Boarding has been developed and used. Today’s Mega Project Directors, must quickly be able to see how a process will play-out and also see how mitigation of Project Risk variation is considered. Story Boarding is also a great tool for problem solving.

Competencies, Indoctrination and Training: A most important aspect in the overall scheme of management on Mega Projects is the need to enhance individual competencies with indoctrination and training. Management must provide continuous training for all project participants. The word “indoctrinate” is the perfect word for this activity, however, these days, many are put off by it’s use, thinking it is too strong. Indoctrinate means: to instruct especially in fundamentals or rudiments. Often project participants, especially commercial and legal colleagues on the project, have little to no indoctrination in the principles of management systems. Make Indoctrination and Training an mandatory part of project introduction for new employees, especially Commercial and Legal personnel.

Lessons Learned Syndrome: We hear a lot today about Lessons Learned, how we can learn from our past experiences, our past failures. When something is planned to be done in a specified way, based on some level of confidence and competence, and the end result is not as planned/expected, we can then say we have Learned something. Too often, however, we are seeing activities being performed in an unplanned manor, with no consideration as to confidence or competence to be applied to achieve the desired results. Often the expected end results are not even known. It’s like “point, shoot, then aim”. When this results in a failure, and in most cases, it is destined to fail, management calls it “Lessons Learned”. To this my company says – “Lessons Learned is Money Burned”. Money is wasted doing something wrong from the beginning. There was no lesson learned here, just money burned/wasted. When this practice continues, the management suffers from what we call - “Lessons Learned Syndrome”. Only if there is a realization that this style of management will not work, and to this a planned and systematic approach is adopted for future activities, can a Lesson be Learned. Avoid the “Lessons Learned Syndrome”. Plan it?, Do it Right and remember - To do it Right, the Doer must know how to do it – and be empowered To Do It. If not , then find out how to do it, before attempting to do it. We find there is never enough time to do it right, but time is always found to do it over and over again.

Ultimate Customer Satisfaction: ISO-9000 ushered in the current era for Improvement and Customer Satisfaction. Ever since, companies have been vigorously working to improve themselves and doing more to satisfy their customers. Their enhanced quality management system then becomes the focus for achieving excellence for their “Company”. As they work to achieve these internal improvements, the focus is still on “self-excellence”. Typically Projects have a fixed cost and fixed schedule. Quality improvements are normally looked for as cost reduction efforts finding ways of reducing the cost of doing something on the project, either by improved processes or products, without compromising end quality. Many, especially Clients, see this still as “self-excellence”. “Partnering” initiatives have been found as a means to bring the Clients and Contractors closer together. “Client/Customer Relations Management” focus on Customer Satisfaction. The most successful projects are those projects on which Successful efforts and commitments have been made to improve Client – Contractor relations by truly “Partnering”. This only evolves when the stereo-typical, Client-Contractor adversarial project management style is transformed. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 16 of 18

We have started a unique concept for project improvement and client satisfaction. Instead of Contractors focusing on “Self-Excellence” as a way towards Customer Satisfaction, turn it around an focus on – “Achieving Excellence for your Client”. Empowerment must be felt at all levels of the Project. All must be empowered to say, NO!, when it’s not being done Right. Project Directors will transform from an Autocratic Project Management style to an Empowerment style. Make your mission, your only purpose, to Achieve Excellence for your Client. If your Client is considered as Excellent, you then have the Most Satisfied Customer.

Conclusions

Managing Mega Projects is transforming. Mega Projects are now “Co-Piloted” by Lender and Legal professionals, making Risk based decisions. System Integration Management indoctrination must start from Project Conception. Independent “Multi-Level Checking” is here and will remain until the System Integration and Assurance Management Systems can demonstrate the confidence needed by the Lenders and Lawyers. Strive to achieve Excellence for your Client and you have achieved the ultimate in Customer Satisfaction. Demands for Technical Compliance are being replaced by demands for “Management Compliance”.

References

ƒ CENELEC EN 50126, Railway Applications – The Specification and Demonstration of Reliability, Availability, Maintainability, and Safety (RAMS) ƒ CENELEC EN 50129, Railway applications; Safety related electronic systems for signaling ƒ EU Interoperability Directive 96/48, The Technical Specifications for Interoperability (TSI’s) for cross border high speed rail lines ƒ Integrating Management Systems for Projects current practices; ƒ ISO-9000, Quality management systems - Fundamentals and Vocabulary ƒ ISO-9001, Quality Management System - Requirements ƒ ISO-9004, Quality management systems -- Guidelines for performance improvements ƒ ISO-19011, Guidelines for quality and/or environmental management systems auditing ƒ ISO-14001, Environmental management systems - Requirements with guidance for use ƒ Past practice on Metro Mega Projects: Taipei Metro; Athens Metro; Bangkok Metro; Ottawa Lite Rail; Tel Aviv Lite Rail; Lisbon Lite Rail; Porto Rico Lite Rail; Taiwan High Speed Rail; Korean High Speed Rail; Holland High Speed Rail Link (HSL); ƒ Project Management Institute (PMI) - A Guide to the Project Management Body of Knowledge (PMBOX® Guide) ƒ SOX (Sarbanes-Oxley Act 2002) Compliance; ƒ INCOSE Systems Engineering Handbook v3.1 (9/1/2008) ƒ INCOSE Metrics Guidebook for Integrated Systems and Product Development (8/31/2007) ƒ INCOSE Systems Engineering Leading Indicators Guide (6/15/2007) ƒ INCOSE Systems Engineering Measurement Primer ((8/1/1997) ƒ INCOSE Systems Engineering Vision 2020 (2/23/2009) ƒ INCOSE Technical Measurement (12/27/2005)

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 17 of 18

Biography

The Author has been an innovator for integrating the various management systems found on major infrastructure projects (Over $1 Billion) in SE Asia (Taiwan, Thailand, Korea), Europe (Holland, Greece, France, Germany, Portugal), North America (Miami, Chicago, Ottawa, Porto Rico). He created and tested a System to Integrate the various Management Systems of Transportation Mega-Projects that are applicable to any Project. The System, called Integrated Management System for Projects® (IMSP) ®, is applied to a Project, to provide Management, Stake Holders, and Overseers, a real-time Scorecard as to Project Risks and other critical Processes. His early exposure to System Engineering/Integration and Assurance Management was in aerospace research & development testing for manned flight spacecraft, where the focus was on design & safety requirements. He then saw how RAMS (reliability, availability, maintainability, safety) plays a vital role in life cycle costs for avionics in the airline industry. He came full cycle, landing him back on the ground, in a passenger rail car manufacturing company. For over 100 years, this company had been building passenger rail cars for intercity and metro lines in the States. Pride in personal craftsmanship was passed down from father to son. In the 70’s however, the company was facing market share loss, due to it’s outmoded production methods and labor costs. Manufacturing costs and the amount of rework on the line were killing the company. They decided to look for improvements. It was too late - They died. Dade County, Florida was building a “high profile” Metro Rail System in Miami, and he got involved, in the 80’s on his first Transportation Project from the Clients point of view. The challenge presented was to establish a management system to bring all the processes of the Transportation System together. Using some of the basic principles/practices of System Engineering/Integration learned while in the Aerospace industry from various standards that existed at the time for adaptation on the Miami Metro Rail Project. Working for the Public Sector brought new challenges to his thinking for integrating SE, QM, RAMS and PM and the challenging role for a Systems Integrator. At times, he found himself in conflict with what was learned on the private side. Blending started. In Taiwan, the Taipei Metro Rail Transit System was being planned. In 1989, Bechtel, the a General Consultant Partner, hired him to direct Project System Assurance (QA, RAMS, H&S, VE, HF). His first true “Mega Project”! The Taipei Metro has since been voted the “Worlds Most Reliable Metro” for the last 6 years running. Taiwan was very keen to implement an integrated management system. Multi-national companies were brought together to bid and build the Taipei Metro Project. Multi-cultural impacts began to appear. Not only national cultural differences and impacts - but also corporate cultural differences and impacts. More lessons learned and the blending continued. Similar impacts were observed when assessing the management systems being proposed for the Korean High Speed Rail Project and the Taiwan High Speed Rail Project. The Taiwan High Speed Rail Project introduced new elements into the systems for managing Mega-Projects. One was, the Design, Build, Operate concept, whereby the “Concessionaire” not only designed and built the Project, they also must operate it and take profits from the operation. Funding for the project was initially to be solely from the Concessionaire, but after risk balancing of life cycle costs, the Taiwan government stepped in to back the Concessionaire. The second element introduced was “Multi-level Independent Checking for Verification and Validation”. Designers had to hire “Independent Checking Engineers” (ICE) to verify and validate designs. Constructors had to hire “Independent Site Engineers” (ISE) to verify and validate construction installations. Lastly, an “Independent Verification and Validation” (IV&V) company was hired to independently verify and Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 18 of 18 validate the entire project and report to the Taiwan Ministry for Transportation. Checkers – checking Checkers – checking Checkers – checking Checkers. What’s happening? Owners, and more importantly, Lenders, are demanding more risk protection and a more balanced allocation of project risk. Life cycle cost impacts lead to risk avoidance efforts. The ability to effectively integrate management systems of a Project is now seen as a major risk. Confidence in quality management systems for major projects has eroded and are being challenged. Why? Fragmented Management Systems, duplication of activities, failure in Interface Management Systems and a general lack of transparency and seamless operation between Project Participants. The Author has consulted, advised and directed activities on Mega-Projects with such companies as Siemens AG, Bechtel International, McDonnell-Douglas, General Dynamics, Rockwell, Pullman, Westinghouse, General Railway & Signal, Union Switch & Signal, Knorr, Bosch, Alsthom, Alcatel, Cegelec, Segole, Spie Batignet, Otis, Hochtief, D-E Consult, Kawasaki, Mitsubishi, Cable & Wireless, ABB, Goldstar, Daewood, Hyundai, AEG / ADtranz, Parsons Brinkerhoff, Kaiser Engineers, Deleuew Cather, Booz-Allen, DE-Consult, Hamburg-Consult, and Fluor-Daniel.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 12

Using Systems Engineering to Implement Corporate Social Responsibility to Meet the Challenges of a Globalized World

Professor Annik Magerholm Fet Norwegian University of Science and Technology Department of Industrial Economics and Technology Management NO 7491 Trondheim, Norway Office: +47 73 59 35 09, [email protected]

Copyright © 2009 by Annik Magerholm Fet. Published and used by APCOSE with permission.

Abstract. This paper gives an overview of environmental management and related tools, corporate social responsibility, and systems engineering. Environmental management techniques are presented from three different perspectives; sites, life cycle and global value chain. Corporate social responsibility (CSR) is just one of many ways that firms are addressing the human dimension in their conduct of business. The paper offers a short introduction to systems engineering as a methodology to approach the challenges businesses are facing in their efforts to implement CSR-strategies.

Introduction Topics such as industrial ecology, environmental management, corporate social responsibility, global value chains and systems understanding are becoming more interconnected as business are going global. The international conventions - the Global Compact (UN, 1999), the UN Millennium Development Goals, and the Global Reporting Initiative (GRI, 2002) – provide important guiding principles for business that want to meet the sustainability challenges. As a result of globalization the focus of corporate social responsibility (CSR) needs to be widened. For companies that want to improve their environmental performance, environmental management tools and techniques are crucial. They can be applied to different levels of systems; for a corporate site, for a product’s life cycle, and to the global value chain. This requires multidisciplinary understanding and systems thinking.

Industrial ecology (IE) and corporate social responsibility (CSR) Industrial Ecology is the broad umbrella or the framework for thinking about and organizing production and consumption systems in ways that resemble natural ecosystems. This idea considers human societies to be part of and operating within natural ecosystems (Ehrenfeld, 1994). The aim of IE is to interpret and adapt an understanding of the natural systems and apply it to the design of the man-made systems, in order to achieve a pattern of industrialization that is not only more efficient, but also is adjusted to the tolerances and characteristics of the natural system. The emphasis is on forms of technology that work with natural systems, not against them. The concept of industrial ecology was first introduced in 1989, with the publication of an article in Scientific American entitled “Strategies for Manufacturing” by Robert Frosch and Nicholas Gallopoulos (in Ehrenfeld, 2004). Simply stated, an industrial ecosystem optimizes the consumption of energy and

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 12

raw material while minimizing the creation of waste and pollution by creating a use for everything produced in a manufacturing process – both desired and waste products.

There are several definitions of IE (O’Rourke et al, 1995), and they all take into account objectives such as closed material cycles, evolutionary principles, resiliency of systems thinking (the ability to recover), dynamic feedback, and cooperation and competition in ecosystems. The definition by Graedel et al. (1995) is based on a system view:

Industrial Ecology is the means by which humanity can deliberately and rationally approach and maintain a desirable carrying capacity, given continued economic, cultural, and technological evolution. The concept requires that an industrial system be viewed not in isolation from its surrounding systems, but in concert with them. It is a system view in which one seeks to optimize the total material cycle from virgin material, to finished material, to component, to product, to obsolete product, and to ultimate disposal. Factors to be optimized include resources, energy, and capital.

Industrial ecology operates at three levels, at the firm level, across firms and at a regional or global level. At each of these levels, industrial ecology aims to provide tools and knowledge for analysis and design towards more sustainable solutions. At the firm level we find green accounting and environmental management; across firms, industrial symbiosis, product-life cycle management, and supply chain management; and, at a global level, material flow analyses and other tools.

Corporate Social Responsibility (CSR) is about business and industry taking responsibilities beyond that of creating economic value for shareholders. For companies to make an honest contribution to CSR, it is important that they have a clear understanding of what it really means for them. There is not yet a universal definition of the concept of CSR, and the concept is still vaguely defined (Michael, 2003). According to Marrewijk (2003) CSR is too often seen as “the panacea that will solve the global poverty gap, social exclusion and environmental degradation”, and he believes that it is necessary to develop a clear definition before working further with the concept of CSR.

Dahlsrud (2008) has evaluated the existing CSR definitions to determine the definition that incorporates the most important concepts of CSR. The study showed that the definition of the European Commission (2001) was most used in the discourse, and as a result it was seen as the most comprehensive definition. Their definition is cited here (European Commission, 2001):

“…a concept whereby companies integrate social and environmental concerns in their business operations and in their stakeholders on a voluntary basis “

The definition of the European Commission (2001) includes both social and environmental dimensions, and the phrase “business operations” is an expression of the economic dimension. Other important dimensions are stakeholders and voluntariness. Dahlsrud’s analysis (Dahlsrud, 2006) found that these five dimensions are widely used within the definitions, and that they are regarded as important components of CSR.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 12

Sustainable development and environmental management The three dimensions – social, economic and environment – are referred to as the triple bottom line, and also as the three pillars upon which the concept sustainable development rests. For many, CSR is mainly concerned with the social aspects. In the larger picture, CSR is also about how companies handle the economic aspects, which includes how they create value for their shareholders, and last, but not least, the environmental aspects. This last is the focus of this paper.

Figure 1 illustrates the progress towards sustainability mainly addressing the environmental challenges. The horizontal axis shows the temporal concerns; the vertical axis illustrates the scope of environmental concern. The different numbered blocks represent different approaches to environmental consciousness. Block 1 represents environmental engineering with a focus on the manufacturing process of a product’s life cycle. Block 2, pollution prevention, also considers the planning phase, while 3, environmentally conscious design and manufacturing, emphasizes the entire life cycle of a product. As we see, block 4, Industrial Ecology, encompasses several products and manufacturers over a long term perspective, and finally 5, sustainable development, has a broad perspective concerning the whole society in a long term perspective that is also multi-generational. The challenge is to move from the lower left corner towards the upper right corner.

5 Society

X Manufacturers One Manufacturer 4 X Products e Disposal 3 The challenge: Move from the Use lower left towards the upper right corner. 2 1 Manufacturing Single Product Life Cycl Life Product Single Planning Manufact. Use Disposal Human Civilization Life time Span time Scope of Environmental Concerns System/Product Life time

Scope of Temporal Concern

Figure 1: Progress towards sustainability (Fet, 1997, modified after Bras, 1996).

To help companies in meeting these challenges, several standards are already available. The most recognized for environmental management are the ISO 14000-standards. Some

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 12

of these concern the organization; others are for environmental product assessment. It can be very chaotic for companies when they start looking at possibilities and tools for improving their environmental performance. The chaotic picture can be made more systematic for companies by sorting the tools into process-related, product-related and management related approaches, as seen for some of the environmental related assessment tools, see Figure 2.

Cleaner Production (CP) Environmental Accounting (EAc) Process related

Life Cycle Assessment (LCA and LCC) Material, Energy and toxicity analyses (MET) “Material Input per Service Unit” (MIPS) Product related Design for the Environment (DfE)

Environmental Auditing (EA) Environmental Performance Evaluation (EPE) Management related Environmental management Systems (EMS

Figure 2: Methods and tools categorized

Another way of systematizing the tools is by using the framework presented in Figure 1. Figure 3 uses the same horizontal and vertical axis to illustrate how process related, product related and management related tools can be applied systematically to meet the challenge of sustainable development.

In cleaner production and in environmental accounting, site-specific input-output analyses have been the traditional approach. Input-output analyses looks at material and energy flows into a production system, and emissions or pollution to air, water and ground as outputs. Changing the viewpoint to a global scale, the same analysis is still important, but now with a wider focus considering the opportunities for improvement along the entire value chain also including sub-suppliers and the consumers. The product related tools, see Figure 3, concern more than one manufacturer, they represent a shift from the site focus to a focus on a larger system - the entire value chain of the product consisting of the production phase, the use phase and the end of life phase. Input-output analyses from each of these phases are needed for a complete life cycle assessment (LCA) of a product.

The results from an LCA are often presented by means of performance indicators showing the contribution to different environmental impact categories as shown in the model in Figure 4. The relative contributions to the different impact categories show the hot-spots and potentials for environmental performance improvements of a product, illustrated by the diagrams at the right in the figure. Information about the product can be presented in environmental product declarations (EPD) (ISO 14025, 2006, Fet and Skaar, 2006).

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 12

Policy programs, local, national Society and international (Protocols, Agenda 21)

X Manufacturers

One Manufacturer X Products e EMS, EPE, EA Disposal LCA, LCC, MET, MIPS, DfE Use Scope of Environmental Concerns CP, Manufacturing EAc. Single Product Cycl Life Planning Manufact. Use Disposal Company's Human Life Life time time System/Product Life Cycle

Scope of Temporal Concern Figure 3: Application of process, product and management related tools to meet the challenge of sustainable development (Fet, 1997).

Performance indicator Environmental Input Output Impact categories

NOx Acidification 0,7 0,6 Energy SO2 Eutrophication 0,5 Prod. 0,4 Material CO2 Global warming 0,3 Process 0,2 0,1

xxx Ozon depletion 0 Nitrogen Oksygen Biomasse Gjødsel xxx YY

NOx Acidification 25 20 Energy SO2 Eutrophication 15 Use 10 Material CO2 Global warming 5 0 e rid k var Jern o uk råvare P rå mkl u om Kalkstein lje som Natri xxx Ozon depletion Kull s O

Naturgass som råvare xxx YY

NOx Acidification Energy SO2 Eutrophication 700 End of 600 Material CO2 Global warming 500 life 400 xxx Ozon depletion 300 200 xxx YY 100 0 Kull Naturgass Olje Kjerneenergi Vannkraft

Figure 4: Model of impact assessment in LCA

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 12

Performance indicators and the GRI framework One of the initiatives of the United Nation Environmental Program (UNEP), supported by Worlds Business Council for Sustainable Development (WBCSD), is the Global Reporting Initiative (GRI). GRI was established in 1997 with the mission of developing globally applicable guidelines for reporting on economic, environmental, and social performance. The GRI's Sustainable Reporting Guidelines represents the first global framework for comprehensive sustainability reporting. The latest version (G3) was published in November 2006, which gives guidance to reporters on selecting generally applicable and organization-specific indicators, as well as integrated sustainability indicators. The G3 replace the 2002 Guidelines. The GRI Indicator Framework organizes the performance indicators in accordance with the following hierarchy:

• Category: Under economic issues we find the ones that have a direct economic impact, under social issues we find four categories Labor practice and decent work, Human rights, Society and Product responsibility. • Aspects are the general subsets that are related to a specific category.

Performance Indicators are the specific measurements of an individual aspect that can be used to track and demonstrate performance. These are often, but not always, quantitative. A given aspect (water) may have several indicators (e.g., total water use, rate of water recycling, discharges to water bodies). A pillar of the GRI framework is that aspects and indicators derive from an extensive, multi-stakeholder consultative process.

Under the category Environment in GRI we find the aspects Materials, Energy, Water, Emissions, Effluents, and Waste. These are dealt with within site specific analyses (e.g. input-output analyses). When we shift to a product focus and look at the value chain, or network of actors along the life cycle of the product, aspects such as Suppliers, Products and Services, Compliance and Transport become important in addition to the aspects already mentioned. For each aspect the GRI-framework suggests a set of performance indicators. For these aspects under the category environment, the recommended indicators are listed in Table 1.

Table 1: Aspects and performance indicators for the category environment (GRI, 2002) Aspect Performance Indicators Suppliers Performance of suppliers relative to environmental components of programs and procedures described in response to Governance Structure and Management Systems section Products and • Significant environmental impacts of principal products and services. Services • Percentage of the weight of products sold that is reclaimable (recyclable or reusable) at the end of the products’ useful life and percentage that is actually reclaimed. Compliance Incidents of and fines for non-compliance with all applicable international declarations / conventions/treaties, and national, sub-national, regional, and local regulations associated with environmental issues. Explained in terms of countries of operation.

The emerging international guidelines for social responsibility (SR), the ISO 26000 standards, will build upon the same model as the ISO 14000-standards, following the plan-do-check-act improvement cycle. Usage of ISO 26000 will be voluntary. It will not

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 12 include requirements and will thus not be a certification standard. However, it will provide practical guidance related to operationalizing social responsibility, identifying and engaging with stakeholders, and enhancing credibility of reports and claims made about social responsibility within an organisation. This means it will have an influence on improvements in the value chain. Nevertheless, it will not be a standard for assessing CSR-aspects related to the product life cycle.

Similar models, such as those used for environmental assessment, could be used to address the CSR-issues at the site and in the product value chain. The challenge is to do this in a coherent way, e.g. by standardized methods. The life cycle impact assessment methodology could be further developed to include indicators for a set of actual CSR-impact categories.

In the GRI-framework, the social issue has 4 categories. Similar as for environmental issues there are defined a set of aspects and related performance indicators. This is demonstrated for the aspect Product responsibility in Table 2.

Table 2: Examples of performance indicators for the category “product responsibility” under the social issues (GRI, 2002) Aspect Performance Indicators Customer Health • Description of policy for preserving customer health and safety during use of and Safety products and services, and extent to which this policy is visibly stated and applied, as well as description of procedures/programs to address this issue, including monitoring systems and results of monitoring. • Number of complaints upheld by regulatory bodies to regulate the health and safety of products and services. • Voluntary code compliance, product labels or awards with respect to social and/or environmental responsibility. Products and • Description of policy, procedures/management systems, and compliance Services mechanisms related to product information and labeling • Number and type of instances of non-compliance with regulations concerning product information and labeling. • Description of policy, procedures/management systems, and compliance mechanisms related to customer satisfaction. • Number and types of breaches of advertising and marketing regulations. Advertising/ • Description of policy, procedures/management systems, and compliance Respect for mechanisms for consumer privacy. Privacy • Number of substantiated complaints regarding breaches of consumer privacy.

By comparing the indicators in Table 1 and Table 2, we see the following: For environmental issues the listed performance indicators address upstream requirements: • Requirements on the performance of suppliers: o On the practicing of environmental management o On compliance with external regulations

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 12

• Requirements on the supplier regarding product information: o On the environmental impacts from products o About the recyclability of products (end-of life treatment)

For the products responsibility category under the social issue, the listed performance indicators address obligations downstream (to customers): • Obligations concerning open information about o Self-imposed procedures and codes of conduct o Internal systems to monitor the procedures o Openness about complaints and breaches of good practices • Obligations concerning openness about product information on o Potential health aspects from products o Eco-labeling and implemented systems for providing such on own products

These requirements are summarized and illustrated by Figure 5.

Environmental issues: Social issues: Requirements to performances Obligations to provide information upstream: Company downstream (to customer) about: - suppliers - management procedures - products - products potential implication

Requirements Obligations

Figure 5: Illustration of requirements regarding environmental issues, obligations regarding social issues, according to the GRI-system.

Companies that have implemented an environmental management system are committed to require information about the environmental performance of each supplier, and also to use this information as criteria when they choose their suppliers. This is also one of the requirements in ISO 14001. In the last version of ISO 14001 (ISO 14001, 2004) the requirement to identify environmental aspects concerning their products, processes or activities is now changed to products, processes and activities This means that the focus on the environmental aspects of the products has been strengthened.

Systems engineering, CSR-management and the progress towards sustainability Systems Engineering (SE) offers an approach for companies trying to use different standards for similar purposes. Figure 6 illustrates the SE approach in 6 steps (Fet, 1997).

Based on requirements from the “customers” (a user group or a market), a need is identified. This step includes an iterative loop where the statement of needs is an answer to the question “What is needed?”, the logic is an answer to the question “Why is it needed?”

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 12 and the search for solutions answers the question “How may the need be satisfied?”. The “statement of need” should be presented in specific qualitative and quantitative terms, in enough detail to justify progression to next step.

Step 1. Customer's Identify Needs Requirement

Step 2. Additional Research Define Requirements

Step 3. Specify Performances

loop Step 4. feed-back Analyze and Optimize

Step 5. Design and Solve

Step 6. Verify and Test

Figure 6: The Systems Engineering methodology (Fet, 1997).

In the context that is described in this paper, especially on the obligations the companies have vis-à-vis the customers, answers to each step in Figure 6 can be formulated in the following way:

STEP 1 The needs identification delivers product information / declaration, and company presentation / information in a coherent and consistent way.

STEP 2 Definition of the requirements should produce clear communication and verifiable information that answers the questions • Which performance indicators shall be included? • Which set of impact categories should be included?

STEP 3 It is important to be able to follow up on the performance and to benchmark/compare the information between different alternatives. This step answers the question, ’How can the information be presented in qualitative and quantitative forms?’

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 12

STEP 4 The information / indicators/ categories should be analyzed for different systems and purposes. System modeling of global value chains and use of analytical tools are be important here.

STEP 5 As part of designing and implementing a solution, generate an optimized set of performance indicators and information declarations (e.g. EPDs with added information on health impact, social aspects) and their relevance to other reporting systems.

STEP 6 The final step concentrates on processes that verify and validate the needs defined in step 1 (verification procedures, criteria etc.), and related testing procedures in accordance with international expectations and future standards.

The SE-methodology builds upon the principles of feedback, and this is the same in most of the management standards; the principles of continual improvement are fundamental. The six-step methodology can therefore be illustrated by a cyclic model as illustrated in Figure 7. For each box, some guiding principles for CSR-management will be derived. The intention behind the guiding principles will be to help companies to meet their obligations as illustrated by Figure 5, at the same time as they are using similar principles to define the requirements to their suppliers, as shown in the same figure. A similar model can be further developed on a larger system level, for a network of suppliers, producers and customers.

Step 1: Identify the .-.obligations to deliver information needs

Step 6: Verify .-.verification criteria and test and.proceduresStep 2: Define .-.clear communication the requirements (indicators)

Step 5: Design .-.EPD / CSR-reports and solve Step 3: Specify .-.quantification of information the performance Step 4: Analyse adn optimise .-.benchmarking

Figure 7: SE as a CSR-management model

Another way of seeing CSR-management as a tool towards sustainable development is shown in Figure 8. Using the same axes as Figure 1 and Figure 3, this diagram shows the progress towards sustainability in the context of the application of the methods and tools. As illustrated in Figure 8, CSR and SE can be placed within the same framework.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 12

Policy programs, local, national and Society international (Protocols, Agenda 21)

SE X Manufacturers CSR

One Manufacturer X Products e EMS, EPE, EA Disposal LCA, LCC, MET, MIPS, DfE Use Scope Environmental of Concerns CP, Manufacturing EAc. Single Product Life Cycl ProductSingle Life Planning Manufact. Use Disposal Company's Human Life Life time time time System/Product Life Cycle

Scope of Temporal Concern Figure 8: CSR-management and SE Discussion and conclusion This paper has pointed out some of the sustainability challenges business are facing in the context of globalization. With the focus changing from the traditional site-based concerns to a concern for the entire value chain of products and services there is a need to apply new methodologies and standards. Input-output analyses and environmental management standards have been in use for some years and businesses are becoming familiar with these tools. Waste management systems and energy saving programs are examples of these. However, when the business responsibilities widen to larger systems involving different actors along the product value chain, businesses are meeting new challenges regarding the way they are able to communicate their performance, both on the managerial level and about their products and services. A further challenge is how to communicate this downstream and at the same time require the same information from upstream sources. Tools for LCA and guidelines on sustainability reporting and EPDs are already available and in use in larger companies. Smaller companies are still struggling with how to find the best ways for communicating their performances relative to the triple bottom line.

This paper has suggested using Systems Engineering as an approach to gather the needed information for this communication. However, this will be further dealt with in future research programs, such as reported in Haskins (2007). To sum up, the challenge towards sustainable development has been met by both the governmental agenda and the business agenda. However, throughout the last 30 years we have seen a shift from governmental command and control through co-regulatory principles, to economic instruments and partnership. On the other hand, business has moved from compliance and cleaner production to improvement of eco-efficiency and now putting CSR on their agenda and taking a larger systemic responsibility on the way to sustainable solutions.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 12 of 12

References Dahlsrud, A. 2008. How corporate social responsibility is defined - An analysis of 37 definitions. Journal of Corporate Social Responsibility and Environmental Management 15, 1–13. GRI, 2002. Sustainability Reporting Guidelines, http://www.globalreporting.org/guidelines/2002.asp Ehrenfeld, J. 2004. Industrial ecology: a new field or only a metaphor? Journal of Cleaner Production 12, 825-831. Ehrenfeld, J. 1994. Industrial Ecology: A Strategic Framework for Product Policy and Other Sustainable Practices. Green Goods: The Second International Conference and Workshop on Product Oriented Policy, Stockholm, Sweden, 1994. Fet, A. M. 1997. Systems engineering and life cycle performance within ship industry. ITEV-report 1, NTNU, Norway, 1997. Fet, A. M. Skaar, C. 2006. Eco-labeling, Product Category Rules and Certification Procedures Based on ISO 14025 Requirements. International Journal Of Life Cycle Assessment 11(1):49-54. Graedel, T.E. 1995. Allenby, B.R., Industrial Ecology, Prentice Hall, USA. Haskins, C. 2007. A Systems Engineering Framework for Eco-Industrial Park Formation. Systems Engineering 10: 1, 83-97. ISO 14025, Environmental Labels and declarations, Type III environmental declarations, Principles and procedures. 2006 Marrewijk, M. 2003. Concepts and definitions of CSR and Corporate Sustainability: Between Agency and Communion. Journal of Business Ethics, Vol.44. Michael, B. 2003. Corporate Social Responsibility in International Development: An Overview and Critique. Corporate Social Responsibility and Environmental Management 10, 115-128. O'Rourke, D., Connelly, L., Koshland, C. 1995. Industrial Ecology: A Critical Review, Draft-November 3, 1995, University of California, Berkeley, USA. UN Global Compact 2005. What is the Global Compact? http://www.unglobalcompact.org/Portal/Default.asp?

Biography Dr. Annik Magerholm Fet is a professor at the Norwegian University of Science and Technology (NTNU) where she is an expert in environmental management, systems engineering and life-cycle analysis. She currently leads multidisciplinary initiatives within NTNU for Global Production and Corporate Social Responsibility. She is also the Director of Education and Research for the Norwegian Systems Engineering Council and holds a position as board member of the Environmental Product Declaration (EPD) Norway.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 9

DSTA’s Journey in Systems Architecting

Tan Yang How, Teo Siow Hiang, Lee Keen Sing and Lim Hang Sheng Defence Science and Technology Agency 1 Depot Road, Defence Technology Tower A Singapore 109679 {tyanghow, tsiowhia, lkeensin, lhangshe} @dsta.gov.sg

Copyright © 2008 by Tan Yang How, Teo Siow Hiang, Lee Keen Sing and Lim Hang Sheng. Published and used by APCOSE with permission.

Abstract

In recent years, Systems Architecting (SA) has been a topic of interest among government organisations, academia and the industry. The SA journey for the Defence Science and Technology Agency (DSTA) began in late 2003 with the appointment of three Systems Architects with the charter to discover and exploit new capabilities to support the Singapore Armed Forces. DSTA is a statutory board under the Ministry of Defence with the responsibility to harness and exploit science and technology for the defence and security of Singapore.1 This paper presents an organisational perspective on SA and how it may apply in the local context with an architectural framework. Key lessons learnt in the past few years applying SA methodology to realise System of Systems capability are also highlighted.

Introduction

The development of defence systems has become increasingly larger in scale and in complexity. The advancement of communications, computing and information technologies has enabled individual systems to inter-operate, resulting in network-centric System of Systems (SoS). Over the past few years, there is a growing interest to develop the Systems Architecting (SA) discipline to examine SoS issues. SA is considered both an art and a science (Maier and Rechtin 2000) to realise SoS capabilities. SA is an art because the solution is often derived from intellectual discussions and engagements with key decision-makers and domain experts. It is a science because it uses architecture as a tool for addressing global integration, consistency and integrity in design (Tan et al 2006).

The initiative to transform the Singapore Armed Forces (SAF) into a Third Generation SAF provides the imperative for DSTA to develop SA competency to support this new thrust of transformation. The systems for a Third Generation SAF will be increasingly complex, versatile and intertwined as components of SoS. Through SA, DSTA hopes to be better in leveraging disruptive technologies and developing new defence concepts. SA is one of the means to facilitate the conceptualisation and design of a coherent network-centric SoS capability. The SAF will be able to achieve greater potency in the battlefield with the synergy gained from integrating different platforms, sensors and weapons at the SoS level guided by a robust operational concept (Tan et al 2006).

SA can serve as both an enabler and a catalyst to support the SAF’s transformation journey. As an enabler, it identifies the necessary conditions for us to embark on the journey and to fuel the journey itself. As a catalyst, it stimulates us to review the way we think, organise, equip, train and prepare for future challenges. As an ongoing process, it creates new relationships, and through matching, balancing and compromising proposed functions and forms, it helps to enhance or create new operational concepts to meet the challenges of a changing environment (Tan et al 2006).

The DSTA’s SA journey began in late 2003, when DSTA appointed three Systems Architects with the charter to discover and exploit new capabilities that could support the SAF. Recognising that defence capabilities are the output of complex SoS, the DSTA

1 As the executive agent for MINDEF, DSTA's roles and functions are to: acquire weapon systems for the Singapore Armed Forces; advise MINDEF on all defence science and technology matters; manage defence research and development; design, develop and maintain defence infrastructure; provide engineering and related services in defence areas; promote and facilitate defence science and technology development in Singapore.

Masterplanning and Systems Architecting (DMSA) Programme Centre was subsequently set up in 2006. The role of DMSA is to focus on developing SoS architectures that will provide systems level coherence, fit, consistency and flexibility for the SAF and to spearhead the build-up of this strategic systems level competency. Coherence refers to the different component systems working with each other without interference, conflict or degradation. Fit refers to the system's fulfillment of the SoS's intent and capability with respect to the SoS performance throughout its life cycle - a purposeful alignment with capability master plans, interoperability requirements, technical standards and guidelines. Consistency refers to consistent usage of assumptions such as area of operations, scenarios, systems characteristics and environment. Flexibility allows the SoS architecture to determine the best combination of systems to maintain optimal overall performance despite the changes in dynamics.

System-of-Systems Characteristics and Challenges

In general, an SoS can be defined as an interoperating collection of component systems that produce results unachievable by the individual systems alone (INCOSE 2006 Systems Engineering Handbook). The elements that make up a component system may include the system itself, the personnel to operate and maintain the system, materials used by the system, facilities and information.

An SoS is usually complex and large scale with many interacting and inter-dependent component systems. It is dynamic in nature where interactions between component systems may reinforce for mutual benefits, or lead to catastrophic failure; thus making SoS difficult to understand, analyse and verify. An example of a complex SoS is the integrated air defence network that comprises early warning aircraft, fighter jets, ground surveillance radars, and surface-to-air missile systems supported by a command and control network. An SoS is simply too complex to be treated by quantitative engineering analysis alone (Tan et al 2006).

The five principal characteristics of an SoS (Maier 1996) pose significant challenges in the architecting of modern defence systems. The characteristics of operational independence and managerial independence imply that there will be multiple stakeholders who may have different views and objectives. Therefore, it is important to engage all of them and to encourage collaboration so as to achieve the greater capability. These characteristics are likely to result in tradeoffs in component systems that the affected stakeholders would have to accept.

Figure 1. Integrated Air Defence System of Systems

The evolutionary development characteristic necessitates the need for masterplanning and governance to ensure that component systems are interoperable with no duplicate functions, and critical common resources, such as communication bandwidth, are carefully assigned. It poses additional challenges for component systems as there may be a need to make functional changes or upgrades from time to time to meet the greater capability of an SoS.

The emergent behaviour characteristic may result in uncertainty or unpredictable SoS performance. There are genuine concerns of unexpected catastrophic failures (Axelband, et al

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 9

2007). Last but not least, a vast geographical distribution of an SoS implies that, instead of mass and energy transfer, information transfer is the primary means to achieve greater function capability (Maier 1996). Information assurance and network security will be crucial to ensure SoS capability.

Systems Engineering Versus Systems Architecting

The discipline of Systems Engineering (SE) has been adopted, and is commonly practised in the acquisition and developmental projects in the defence industry. The Vee Model is commonly used as a model to illustrate a system life cycle that consists six stages, namely Concept, Development, Production, Utilisation, Support and Retirement (INCOSE 2006 Systems Engineering Handbook).

There are some noticeable differences between SE practices in DSTA and those of SA as shown in Table 1. The scope for systems engineering encompasses the specific user requirements, system design and development, project management, maintenance support and retirement while SA is concerned with operational and systems concept formulation, force structuring and capability development. The traditional SE approach is applied when the space constraints are well defined to develop a system while SA is applied where the solution space is much larger and operational concepts are generally evolving. For example, traditional system acquisition may have to work within the constraints or assumptions such as available power supply, physical space, rules and regulation. Taking an SA approach can facilitate the re-examination of such constraints or assumptions to open up new possibilities and solutions for the desired capability.

Table 1. Systems Engineering versus Systems Architecting

Traditional Systems Systems Architecting Engineering Scope User Requirements, System Operational and Systems Design and Development, Concept Formulation / Force Project Management, Structure & Capability Maintenance and Retirement Development

Stakeholder Usually one major customer Multiple, Interdependent Relationships Emphasis Deals with measurable, Deals with immeasurable, Technical Feasibility & collaboration, heuristics, Design added ilities such as flexibility, adaptability and scalability

Timeframe System Life cycle Multiple, interacting system life cycles

Trade Off System level Enterprise level

In SA, the needs of multiple, interdependent stakeholders have to be addressed. In traditional SE, each system usually has a major stakeholder, i.e. the customer who funds the system acquisition and development. Hence, SA emphasises collaboration among stakeholders towards a common goal. Similarly, SA needs to address multiple, interconnected and evolutionary system life cycles to maintain a robust and coherent SoS architecture.

An SoS is simply too complex to be treated by quantitative engineering analysis, technical feasibility study or design alone. SA is hence employed to help the designer to visualise, conceptualise, plan, create and build such an SoS. It aims to bring together various systems with the purpose of achieving operational capabilities greater than the sum of what each individual system can provide. SA deals largely with non-measurables using non-quantitative tools and guidelines based on practical lessons learnt. In addition, SA seeks to address non-functional attributes, known as “ilities”. Some examples of “ilities” are flexibility,

scalability and adaptability of an SoS architecture (Axelband, et al 2007).

Last but not least, trade-offs are made at the enterprise level for SA instead of at the system levels for traditional SE. In summary, SA deals with a much greater level of complexity and scale due to multiple interacting systems.

The DSTA Architectural Framework

Figure 2 illustrates the architectural framework currently adopted by DSTA (Tan et al 2006). The purpose of this framework is to guide our work to develop a robust, coherent and enduring SoS architecture. Building an effective SA involves innovation and is iterative in nature.

Figure 2. The DSTA Architectural Framework

Input from the strategic, operations and technical perspectives are important ingredients. The strategic perspective considers the political, environmental, social, and technological factors, as well as the strategic intent put forth by key stakeholders. The operational perspective looks into the mission objectives, potential threat assessment, existing capabilities, resources and operational constraints. The technical perspective takes into account the existing technological capabilities and legacy systems, emerging technology and the physical environment. Where necessary, architectural studies using operational analysis, modelling and simulation techniques, as well as experiments, may be conducted to evaluate alternative architectures.

While the strategic, operations and technology domains are important ingredients for SA, it is the creativity of the integrated SA team in exploiting new technologies, organisational and systems boundaries to devise new and realistic concepts of operations that will determine the effectiveness of the SA. SA involves all domain experts collaborating and co-creating the SoS architecture actively. Quality facilitation and effective change management during the SA process to expand systems and organisational boundaries, and to generate dialogue among stakeholders, is emphasised.

The SoS architecture can be described in operational and technical views. In general, design artefacts can aid in effective communication, knowledge retention and managing complexity (Richards et al 2007). The SoS architectural views can serve as a common language for multiple stakeholders to communicate in a consistent manner. The focus is usually on the organisational boundaries and systems interface. More importantly, these views can highlight integration issues among the component systems and become part of a framework to facilitate SoS governance. Governance will play a critical role in effective synchronisation,

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 9

interoperability and management of multiple programmes to realise the SoS capabilities. To date, an Enterprise Architectural Framework for developing command and control systems has been established with a governance process to facilitate the alignment of technical implementation with operational needs (Yeoh et al 2007).

The endorsed SoS architecture will guide the formulation of various Master Plans. These views serve as the blueprint for development of various Operational Master Plans (OMPs) and Engineering Master Plans (EMPs). Approval of these OMPs and EMPs will lead to individual system acquisitions and sustenance plans. The SoS integration and implementation of component systems will be led by the various Integrated Programme Management Teams. Verification, validation and certification efforts of the SoS will serve as feedback to ascertain if the intent and desired capabilities of the SoS have been realised.

The Systems Architecting Process

The DSTA SA process is developed to serve as a guide to kick off the systems architecting activities. It adopts a life cycle perspective and is developed with simplicity and flexibility in mind. The SA process, depicted in Figure 3, consists six steps and is iterative in nature. This iterative nature of the architecting process is expected as the realisation of SoS spans many years; hence, changes in external environment, for example, may warrant a need to re-examine the SoS architecture. The dotted arrows represent the need to refer back to the earlier steps to verify and evaluate the SoS when necessary. This process is generic in nature and can also be extended to different levels of SoS complexity from capability to individual component system.

Certify SoS 6

1 Frame the 5 Issue Realise SoS

2 Develop SoS 4 Alternatives

Finalise SoS 3 Evaluate SoS Alternatives

Figure 3. The DSTA Systems Architecting Process

Step 1 - Frame The Issue

Systems architecting is driven primarily by the user's purpose and needs. A successful system is one where the user's intent is served at an affordable cost within an acceptable period of time. Hence, the first step in the architecting process is to frame the issue. It aims to discover the higher intent of user articulated needs and objectives, and to discover the underlying assumptions, constraints and limitations to form a rich and unified picture to address the issue. This step will facilitate the involvement of necessary stakeholders so that the right issues are addressed. This will require an examination of strategic, operational and technical perspectives to gain a deeper understanding of the matter in hand.

Step 2 - Develop SoS Alternatives

This step is to generate a broad range of alternative SoS architectures to address the capability gap. The emphasis is on the exploration of the solution space and to consider

solutions involving any combination of doctrine, organisation, personnel, training, systems facilities, emerging technologies, rules and regulation. The architecting team can consider factors such as an alternative SoS concept of operations, network connectivity between specific systems, upgrading of existing systems, and/or new systems acquisition and development.

Step 3 - Evaluate SoS Alternatives

This step involves the evaluation of the set of SoS alternatives in terms of performance, robustness, ilities and cost. Software models would need to be developed to represent each SoS architecture. These models may have already been developed during the development of SoS alternatives and may be used for evaluation purposes. In parallel, test and evaluation parameters must be defined so that those important test criteria are built into the models.

During the evaluation process, new insights from the analysis may result in the need to reframe the issue and/or to refine the SoS design. The architecting team is expected to iterate Steps One, Two and Three of the SA Process. Eventually, the outcome of the evaluation is a recommendation of an SoS architecture that has been assessed for its desired attributes for management’s decision.

Step 4 - Finalise SoS Architecture

The output of systems architecting is an endorsed SoS architecture. This SoS architecture is described in terms of architectural views in accordance with the Enterprise Architecture framework and governance guidelines. The documentation will facilitate promulgation, communication, masterplanning and realisation of the SoS architecture.

The finalised SoS architecture will facilitate the formulation of the various capability development plans. These master plans will chart the milestones for capability build-up, resource and training requirements.

Step 5 - Realise SoS

The realisation of the SoS architecture will usually be led by a Senior Programme Manager (SPM) or Programme Director (PD) supported by the Integrated Programme Management Team (IPMT). There will be different programme teams responsible for the acquisition and development of various component systems in the SoS architecture. Where necessary, a Technical Working Group or SoS Steering Committee may be formed to provide management guidance to the IPMT.

Since each component system will likely have different developmental milestones, the need to work closely among IPMTs to address interoperability and integration issues cannot be over-emphasised. Appropriate SoS Integration Labs may be set up to address integration issues as early as possible using emulators of the component systems.

During the course of SoS realisation, any deviation of the SoS architecture will need to be raised at appropriate governance forums for endorsement. Since the realisation of SoS may take many years, external environment such as disruptive technologies may invalidate the assumptions made during the architecting process. This may result in the need to initiate the systems architecting process again.

Step 6 - Certify SoS

Verification, validation and certification of the SoS are essential activities during this process. Verification and Validation (V&V) is the process to check that the SoS meets specifications and fulfils its intended purpose defined in Step 1 - Frame the Issue. In general, verification is a quality process to ensure that a system complies with specification and one that is conducted throughout the systems development phase. Validation is the process to establish a certain degree of confidence that a system accomplishes its intended mission and addresses stakeholders’ needs.

Both aspects are essential as verification ensures that “we built it right” while validation ensures that “we built the right thing”. Therefore, at this stage, the SoS will be evaluated and

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 9

validated for its capability and performance with respect to the master plan. When the SoS is successfully verified and validated with respect to the master plan, the SA team can proceed to certify the SoS with the customers and stakeholders. It is notable that an SoS may not have a completion date. Unlike a single system which will be developed, fielded and eventually retired, an SoS will need to be enduring to deliver the intended capabilities until it is no longer relevant. Hence, an SoS can evolve through many master plans and renewal of component systems.

The process of V&V may lead to new insights or discovery of undesired emergent behaviour. Lessons learnt will be produced as feedback to the next cycle of the SA process. It is important to have regular monitoring of SoS operations to discover any emergent behaviour. In addition, the operations of SoS must be reviewed in the context of changes in the external environment for deficiency as it may trigger the need for a new cycle of systems architecting.

Systems Architecting Lessons

SA requires a multidisciplinary and integrated team of domain experts comprising operators, systems engineers, software engineers and technologists from various fields working together as a Community of Practice (CoP) to study on a comprehensive range of issues. To achieve this, members of the CoP must be open to new ideas, and be comfortable with expanding boundaries beyond individual domains, to create alignment and commitment so as to collectively achieve a shared vision.

Systems Architects need to know a broad range of subjects and understand enough of each discipline so as to appreciate issues and harness the collective wisdom of domain experts. They must be able to frame the issues in the stakeholders’ context and language so as to enlist the support to resolve these issues. The Systems Architect will need to understand organisational dynamics, be clear about who are the key stakeholders and develop the approaches to resolve differences.

Most of the value proposition of SA can be found at the interfaces amongst systems and trade-offs are necessary steps to achieve the desired SoS capability. The life of the complex SoS can be longer than some stakeholders’ tenure on a particular appointment. Hence, there is a need to consider issues objectively and holistically. From our experience, this is one of the most difficult challenges. Realising SoS is complex, as its design and effective governance are essential to its success.

Last but not least, it is important to adopt a pragmatic approach towards SA. There are many frameworks and methodologies that support and facilitate thinking about issues across people, data, systems and organisations to aid the design and evaluation of SoS architectures.

Conclusion

The research on Systems Architecting is only at her infancy in DSTA and more work is required to develop this competency. An overview of the differences between SA and the traditional SE practices in DSTA has been described. An Architectural Framework and Process for SA has been developed and will evolve as more experiences are accumulated. Finally, some key lessons learnt during the past few years have been shared in this paper.

References

Maier, M. W. and Rechtin, E. 2000. The Art of Systems Architecting, Second Edition. CRC Press.

Tan Y. H., Yeoh L. W., Pang C. K., Sim K. W. 2006. Systems Architecting for 3G SAF Transformation. DSTA Horizons 2006.

Maier, Mark W., Architecting Principles for Systems-of-Systems, 1996 http://www.infoed.com/Open/PAPERS/systems.htm

INCOSE 2006 systems engineering handbook - a guide for system life cycle processes and activities, Version 3. ed. Cecilia Haskins.

Axelband E., Valerdi R., Baehren T., Boehm B., Dorenbos D., Jackson S., Madni A., Nadler G., Robitaille P., Settles S., 2007. A Research Agenda for Systems of Systems Architecting, In proceedings of INCOSE 2007 – 17th Annual International Symposium.

Tan Y. H., Yeoh L. W., Pang C. K., Sim K. W. 2006. Systems Architecting for 3G SAF Transformation. DSTA Horizons 2006.

Richards M. G., Shah N. B., Hastings D. E., Rhodes D. H. 2007, Architecture Frameworks in System Design: Motivation, Theory, and Implementation, In proceedings of INCOSE 2007 – 17th Annual International Symposium.

Yeoh L. W., Lam C. V., Syn H. B. 2007, An Enterprise Architecture Framework for Developing Command and Control Systems, In proceedings of INCOSE 2007 – 17th Annual International Symposium

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 9

Biography

Tan Yang How is Director (Systems Architecting) in the Defence Science and Technology Agency. He is a Senior Fellow at the Temasek Defence Systems Institute at National University of Singapore (NUS). He received his Master of Science in Management of Technology from NUS in 1994 and attended the Program for Management Development from Harvard University in 2004. He was awarded several Defence Technology Team Prizes over the years, including the 1997 Defence Technology Prize (Individual). He led the DSTA effort in developing and fielding the Infrared Fever Screening System during the SARS crisis, for which his team won the Institution of Engineers Singapore Engineering Achievement Award in 2003 and the 2004 Tech Museum Award (Healthcare). He received the National Day Public Administration Bronze and Silver Medal in 2000 and 2003 respectively.

Teo Siow Hiang is Head (Operations Analysis & Simulation). He has a Master of Science for Industrial Engineering (1991), and a Master of Technology for Knowledge Engineering (2001) from NUS. He was awarded the Defence Technology Prize (Classified Project Team) 1990, and DSTA Team Excellence Award (OA & LSA). He received his National Day Commendation Medal in 2005. He is a member of IEEE, MORS, and SCS. Siow Hiang has written, presented and published numerous papers in systems engineering and operations research.

Lee Keen Sing is a Principal Engineer developing systems architectures for complex system-of-systems. Starting as an engineer in the combat vehicles department gaining technical competencies, he had also acquired business competencies during his secondment to the Ministry of Defence as Manager (Capability Development &

Technology Plans). Under the DSTA scholarships, he received his Master of Science in System Design & Management from the Massachusetts Institute of Technology in 2004, Master and Bachelor of Engineering from the Nanyang Technological University in 1998 and 1997 respectively.

Lim Hang Sheng is currently Acting Principal Analyst (Operations Analysis & Simulation). He was awarded the Defence Technology Prize and DSTA Team Excellence Awards in 2001. He graduated with a Bachelor of Engineering and a Master of Science in Electrical Engineering from the National University of Singapore (NUS) in 1995 and 1999 respectively. He obtained a MBA degree from Nanyang Technology University in 2002, a Graduate Diploma in Learning Organisation from Institute of Public Administration Management in 2003, a Master of Science in Defence Technology and Systems from Temasek Defence Systems Institute, NUS and a Master of Science in Operations Research from Naval Postgraduate School in 2007.

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 6

Emerging Behaviors in Large Scale Systems: The Singapore Education System

Ori Sasson School of Information Systems, Singapore Management University 80 Stamford Road Singapore 178902 Tel. +65-68280882 Email: [email protected]

Copyright © 2008 by Ori Sasson. Published and used by APCOSE with permission.

Abstract. The Singapore education system is an example of a large-social system. In post- independence Singapore, the education system was revolutionized in order to ensure economic progress through the creation of a skilled workforce. In this paper we consider three emerging behaviors in the Singapore education system and discuss the impacts of these behaviors and how they were addressed by the Singapore Ministry of Education (MOE). Introduction The Singapore Education System offers an example of a large-scale social system. In post- independence Singapore, the education system was revolutionized in order to ensure economic progress through the creation of a skilled workforce. The idea of Singapore as an example of large-scale systems engineering was highlighted in the first regional Asia-Pacific Systems Engineering Conference, as described by (Sullivan 2007) and (Leo 2007). The Singapore education system is one of the examples cited along with the HDB public housing system, water resource management, transport, and defense development. (Chia 2007) presents the paradigm of large-scale systems engineering at the national level in general and in Singapore. Singapore as a case of large-scale system engineering was also studied in (Chia et al. 2008), and (Lui et al. 2001). The Singapore education system, being a large scale social system is extremely complex. This complexity stems from the large number of stakeholders and participants (including all school- going children in Singapore and their parents in addition to teachers, educational institutions and the government), the long time-cycles (i.e. the long duration each child spends within the system), and the inherent difficulty of predicting the outcome of various design choices. Such complex systems often result in emergent phenomena that could not be predicted by the characteristics of the system or its components. (Rouse 2007) notes that this is particularly true for systems whose subsystems have a degree of autonomy and their own objectives. An education system definitely meets this criterion as each individual student and his or her parents have a certain amount of autonomy. In this paper we consider three emerging behaviors in the Singapore education system and discuss the impacts of these behaviors and how they were addressed by the Singapore Ministry of Education (MOE). The next section provides some background on the Singapore education system, followed by a discussion of each of the emerging behaviors. It is important to note that due to the scale of the education system and the long periods of time involved, it is difficult to quantify the emerging behaviors which are discussed here. Hence the discussion is admittedly oversimplified, and in particular author’s interpretation might be somewhat subjective. Background: The Singapore Education System The two key design goals of the Singapore education system were related to the quantity and quality of the system output. The first goal was increasing the total number of students going through the system, and the second was equipping the students leaving the system with skills which will make them employable, allowing them to contribute to economic progress. Other goals built in to the design of the system were related to ‘nation building’, critical for a young, small, and ethnically diverse nation. The Singapore Education System has shown impressive results. For example, in recent years it produces students who topped international rankings in Mathematics and Science. While this is an achievement by itself, it is an impressive achievement considering that in 1980, less than 30 years ago, only about 58% of Primary 1 students completed secondary school. A key design choice in this education system was the development of a complex set of criteria to ‘route’ each student through different educational options based on his or her performance. This allowed maximizing the potential of each individual student, and avoiding teachers lowering the standard of teaching to the ‘lowest common denominator’. Figure 1 shows the official diagram describing the flow of students through the Singapore education system (taken from the Singapore Ministry of Education website). While a full discussion of the intricacies of the system is not possible due to space considerations, we can note that the key milestones in the system are national examinations. The first of which is the “Primary-School Leaving Examination” administered upon completion of primary school education. Based on the results of the examinations, students are placed in different ‘routes’. The ‘routing’ of students through the system is called ‘streaming’, and was recommended in a report by Dr. Goh Keng Swee in 1978. As described in (Sullivan 2007), Dr. Goh Keng Swee was the first to introduce systems thinking and systems engineering into the Singapore government. As part of the notion of streaming, the different educational options were enhanced to include vocational training through polytechnics and “Institutes of Technical Education” (ITEs), in addition to traditional academic options such as Junior College (JC) and universities. Through the introduction of such vocational training, opportunities were created for every child in Singapore to undergo at least ten years of education. Another two key choices made was to use English and the main language of teaching and teaching students a second language (their own ‘mother tongue’). The choice of English as the main language helped to position Singapore as a regional economic powerhouse. The bilingual policy which allows each child to learn his own ‘mother tongue’ (e.g. Chinese, Malay, or Tamil) in addition to English not only enhances children’s linkage to their cultural heritage, but also increases the ability of Singaporeans to interact with speakers of those languages across the region. (Ng 2008) provides an overview of these choices and the rationale behind them. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 6

Figure 1. The Singapore Education System, taken from MOE Website http://www.moe.gov.sg/education Emerging Behavior: University Education The first emerging behavior is related to the number of students admitted to local universities and their education background. The “original” process for routing students through the system (e.g. 10 years ago) severely limited the number of polytechnic students reaching university. This was due to a cap on admissions of polytechnic graduates to local universities (e.g. 5%). This did not preclude polytechnic graduates from pursuing graduate studies outside of Singapore, or with foreign universities which have presence in Singapore. In addition to this cap, the local job market placed a significant premium on university degrees. A university graduate could earn 30-50% more than a polytechnic graduate upon entering the job market. Such gaps could potentially be retained throughout one’s career, thus making university education an attractive option for polytechnic graduates. The obvious reason is that the one-time investment in education could be amortized over many years of work. The behavior which emerged was that foreign (e.g. Australian) universities started offering “conversion programs” for polytechnic graduates. Such programs would typically run for a single year, and participants would be awarded a Bachelor’s degree, based on course exemptions due to their polytechnic education. For the aforementioned reasons, a growing number of Singaporeans enrolled in these programs. At first the “system” reacted to this emerging behavior by ignoring it. Specifically, the Singapore government and government-owned companies did not recognize the conversion degrees. In other words, holders of such degrees were offered salaries identical to those of polytechnic diploma holders and lower than graduates of local universities. Ultimately, the degrees were recognized. On one hand the government was possibly following the lead of the private sector, but also as it was increasingly difficult to ignore the investment by large number of students. However, with a longer-term view in mind, a serious revision was made in the system by the Singapore MOE. On one hand the number of polytechnic students who can be admitted to local universities was increased. On the other hand, the total number of university-going students was increased (from around 20% to a target of 30% of each cohort). These two measures combined reduce the impact of foreign universities offering graduate programs to Singaporeans, and specifically allow a more efficient use of the resources of Singapore and Singaporeans (e.g. by removing the need for paying for travel and lodging overseas, and by keeping the tuition fees within Singapore). In order to increase the intake of local universities, a new (third) government- funded university was opened 9 years ago and another one is set to be open in the next few years. Emerging Behavior: Widespread Use of Tuition As explained above, the “streaming” system is based on the result of an examination administered at the end of Primary 6, called PSLE. The results of the examination have long- term consequence, as they may impact a child’s education for many years, potentially even having an impact on a child’s ability to receive university education. Other national examinations such as O-levels and A-levels have similar importance. Due to the importance of these examinations, parents in Singapore are increasingly investing in various forms of out-of-school education, or “tuition”, to increase the ability of their children to succeed in the various examinations. Tuition is obviously outside the boundaries of the education system, and the widespread use of tuition is basically ignored by the system. This appears to be a correct choice, especially when Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 6

comparing to other countries. For example, in Korea, school hours were extended in the face of growing use of tuition. Apparently, the rationale was that with long school hours, tuition would not be needed. However the emerging behavior there was that parents created demand for tuition after the longer school hours (e.g. 7pm to midnight). Emerging Behavior: Desire for Diversity The last example of emerging behavior is essentially a result of having a strict system where some stakeholders (e.g. parents) feel that children are not receiving the most suitable education. In earlier times (e.g. 15 years ago), the Singapore education system was somewhat simpler than it is currently. The current system is described in Figure 1 above, and the earlier system is described in Figure 2.

Figure 2. The Singapore Education System 15 years ago Without going into too much detail, the simple and strict streaming concept was not able to cater for the needs and capabilities of all students, and specifically was viewed as a ‘cookie-cutter’ approach. In response to that, MOE has introduced in recent years specialized schools (e.g. Music, Arts, Sports) to allow different students to capitalize on their strengths. It also allowed privately funded schools to have more flexibility in their curriculum. This shows that the system designers are more confident, and thus are willing to tolerate more flexibility in the system. It also shows that the system designers accept a wider definition of success, following the lead of society at large. Conclusion This paper builds upon the view of the Singapore education as a large scale system, and in particular looks at the dynamism in this large-scale system, which reacts to emerging behaviors. The existence of emerging behaviors which cannot be anticipated in full make the system more complex to manage. The three examples discussed herein show that through the use of system thinking, foresight, and flexibility, the policymakers in the Singapore MOE are able to adapt to changing circumstances and requirements. References Chia E.S. 2007. Analysis of Singapore’s 1991 Strategic Economic Plan Using the Large-Scale Systems Engineering Framework. In Proceedings of the Seventeenth Annual International Symposium of the International Council on Systems Engineering (San Diego, California). Seattle: INCOSE. Chia E.S, and Foo G. 2008. Large Scale Systems Engineering Perspective of Water Management in Singapore. In Proceedings of the Eighteenth Annual International Symposium of the International Council on Systems Engineering (The Netherlands). Seattle: INCOSE. Leo, T.B. 2007. Another Look at the Asia-Pacific Systems Engineering Conference 2007. INSIGHT 10 (4): 48-49. Seattle: INCOSE. Lui P.C. and Tan T.S. 2001. Building Integrated Large-Scale Urban Infrastructures: Singapore’s Experience. Journal of Urban Technology, 8 (1) : 49-68. Ng E.H. 2008. Educating the Next Generation. Speech by Minister for Education and Second Minister for Defence, at the 4th Anniversary Public Lecture at Lee Kuan Yew School of Public Policy. http://www.moe.gov.sg/media/speeches/2008/08/14/speech-by-dr-ng-eng-hen- at-the-10.php Rouse, W.B. 2007. Complex Engineered, Organizational and Natural Systems. System Engineering. 10(3) : 260-271. Wiley Periodicals. Sullivan, B. 2007. Lui Pao Chuen – Singapore: An Example of a Large-Scale System. INSIGHT 10 (4): 11-12. Seattle: INCOSE. Biography

Ori Sasson (Ph.D.), is a Practice Assistant Professor at the School of Information Systems in the Singapore Management University (SMU). He has over 20 Years of experience in architecting large scale software systems in the defense, telecom, and utility domains. He is the author of five technical books, and numerous research and practice papers, on diverse topics ranging from Theoretical Computer Science and Bioinformatics to simulation and cultural influences on innovation. He is a Singapore citizen and has been a member of the Singapore chapter of INCOSE since 2007. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 14

The Integrated Design Environment of Luna Probe-Based Software Product Line HU Zhixin LIN Yinming China Academy of Space Technology, Beijing 100094, China Copyright © 2008 by HU Zhixin and LIN Yinming. Published and used by APCOSE with permission.

Abstract

To enhance the core ability of the system engineering technology, and to conquer the difficulties like high system complexity, numerous systems and technology innovation involved, new work mode, severe working environment, pressing project schedule for Luna Exploration Programs, this paper tries to introduce the SPL technology for the design of the Luna Probe Integrated Design

Environment. SPL can support the general design and model research The processing of the Luna

Probe Integrated Design Environment construction is divided into three steps: (1) System Analysis;

(2) System Design; (3) System Implementation.

Key Words: Software Product Line, Luna Probe, Integrated Design Environment

1 Introduction

As a system project of large scale, the Luna Probe faces difficulties such as high system complexity, numerous systems and technology innovation involved, new work mode, severe working environment, pressing project schedule, etc. In order to construct the Integrated Design Environment for the Luna Probe, the following issues should be considered: the task goal of the whole project, implementation steps, various key technical indicators and technical connections, advancements of general design capability and simulation level, as well as supporting the general design and model research. Meanwhile, the construction of the Integrated Design Environment of

Luna Probe can enhance the core technical capability of system engineering, while enriching the tools system used in system engineering.

1

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 14

Combining new requirements with software product line, domain process, and

software architectures in specific domains, all of which are based on commonality

reuse, tailor and fix the specialized parts of the application, thus to realize the

application system. Comparing to other methods, the software product line (SPL)

technology is the large scale, coarse-grained, well planed and from-top-to-bottom

reuse of core software assets for software resources in a specific domain, which

enhances quality and reliability of the development of large scale software systems,

and reduce the development cost. The knowledge in the domain of deep space

exploration has relatively good stability and cohesiveness, thus the SPL technology is

fit for the research in the domain of deep space exploration. Attempting to combine

SPL with the development of the integrated design environment of the Luna Probe,

this paper proposes a method for enhancing the quality and reliability, while

shortening the research cycle and reducing risk and cost of aerospace software

systems, for the development of core assets of deep space exploration and simulation

system. Also, according to the mission requirements of the subsequent deep space

exploration missions (such as the 3rd Phase of the Luna Mission and the Mars

Exploration Mission), we can utilize the SPL to rapidly construct the corresponding integrated design environment by reusing the core assets .

2 The Software Production Line Technology

The software product line (SPL), which is proposed by Software Engineering Institute of Carnegie Mellon University (CMU/SEI) and incarnating the principle of software reuse, is the result of the development of software architectures and software reuse technologies in the software engineering domain, and also is the key reason for a large scale, highly complicated software system to be successful. Software reuse means the large scale, coarse-grained, well planed and from-top-to-bottom reuse of software resources in a specific domain. Currently, SPL is one of the leading aspects of software development. Most of the organizations

2

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 14

that carry out SPL research are universities and large companies, usually sponsored by military. CMU in the United States proposes the SPL model [5][6]; the University of Kaiserslautern in Germany proposes the KobrA Approach [7][8]; the University of Groningen in Holland proposes the COAMOF mode [1][2][9], etc. There are two successful examples of product line systems, which come from the CelsiusTech System Co. in Sweden and the Electronic Systems Center (ESC) of United States Air Force.

This chapter attempts to describe the basic process, construction mode, core

assets development, and the variability of SPL, while emphasizing on the variability

control during the process of developing core assets.

2.1 BASIC PROCESS AND ACTIVITY

The software product line (SPL) model of CMU/SEI, which was developed from

the double-life-cycle model of STARS，has three types of SPL basic activities:(1) core assets development (i.e. Domain Engineering); (2) product development (i.e.

Application Engineering); (3) organization management. The detailed description of

SPL process model and basic activities is shown in Figure 1.

The SPL basic activities include core assets library development and product

development based on the core assets library, both of which need technology

management and organization management. The primary goals of SPL core assets

development (Domain Engineering) are determining the range of the product line,

producing core assets, and defining production plan. While application engineering -

product development based on core assets library - is to produce product and perform

reverse-reengineering. Core assets and product can either be developed at the same

time, or be cross-developed.

3

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 14

Application Domain engineering

Application Requirement

Product Regulation

Style/ Mode/Frame Product Range

Product Core Assets Production Regulation Core Assets Product Development Development

Production Strategy Production plan

List of Existing Assets

Management

Organization Risk Management Management Management

Project Management Process Management

Quality Management Technology Management

Figure 1 Process Model and Basic Activities of SPL

2.2 Development of Core Assets

The SPL core assets are the aggregate of all result products of SPL engineering,

and form the base of product structure in SPL. They include the SPL software

architecture and software components, both of which are the most important parts

used for product development. The software architecture is shared by all products of

SPL. The software components, either obtained from new design/development or

from legacy system reengineering, need to be reused systematically in the entire SPL.

Achieved by the technology of domain engineering, the development of SPL

core assets (Domain Engineering) is the production stage of reusable software assets.

By systematically identifying Commonality and Variability in a specific domain, and

developing/organizing/managing reusable software assets such as Domain Model and

Domain Components, the required base of material and technology can be offered for

4

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 14

the upcoming development of application system.

2.3 Variability Control

Commonality and variability are the two key concepts for domain engineering

and SPL. The character contained in every system on SPL, is the essential character of this product line (PL), and it manifests as the system commonality of this domain; while the character contained in some systems and special applications manifests as the system variability. Commonality is applied to all applications, and its emphasis and activities is turned to choosing and deploying the variability on different levels of abstraction. Variability makes sense only in certain environment of context (i.e. the referencing framework given by commonality), and is not applicable in a context-free environment.

A variable software life-cycle has 5 stages: (1) identification and expression; (2) sort and design; (3) identification of the variability and realization of the variability;

(4) fixing; (5) evolvement, as shown in Figure 2.

Variable life-cycle

Hidden Identified Control Fixing

Variability of identify Variability of sort Variability of identify Variability of and expression and design and realization fixing

Figure 2 The Life Cycle of Variability

To identify, analyze, realize and manage the commonality and variability of a specific domain is the key activity of SPL. In the engineering domain, we identify,

5

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 14

describe and realize the variability, develop domain model and DSSA using the

method of variability control, appropriately distribute commonality and variability of

the domain to architectures and components. The commonality is realized as

architectures, while the variability is realized as components. The interfaces of the

architectures and the components should be as clear and simple as possible, and be

overlapping realized if possible. Based on the requirements of the new system, the

key role of application engineering is to assemble, choose, and fix core assets, realize

variability, and build the application system.

2.4 Development Methods

For the development of SPL, the software organization must have the long-term

strategic vision in order for the determined goal and the normalized process to be successful. Compared to other methods of software development, the macro reasons for choosing SPL are: the clear definition of the product line (PL) and the expert knowledge domain that is required by the PL and its realization; the strategic planning of the long-term prospect of the PL. Based on the import strategy of a software organization, normally there are four methods for the development of SPL, as shown in Table 1: (1) evolving existing products to a product line (PL); (2) replacing existing product set with a SPL; (3) evolvement of a brand-new SPL; (4) development of a brand-new SPL.

Table 1 Mode of SPL Development Incremental Innovatory Developing PL architecture based on Developing PL core assets based on Based on existing product architecture, so as to the requirements of existing product existing gradually develop PL components by set and the superset of foreseeable product set the way of evolving existing upcoming requirements. components. The core assets of product line evolve Developing the PL core assets which Brand-new according to the requirements of new satisfy all requirements of expectant PL product. members of the PL.

6

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 14

3 Development of The Luna Probe Integrated Design Environment That Is

Based on SPL

The question domain knowledge of the domain of deep space exploration has

relatively good stability and cohesiveness, and it is more clearly understood with each

mission. Therefore, with core technologies such as process driven, specific domain,

technology support and architecture, the technology of SPL can effectively advance

the system development quality and reliability of the Luna Probe Integrated Design

Environment, shorten the development cycle, reduce the risk and the cost, support

structural and evolved characteristics of the software system. Also, according to the mission requirements of following deep space exploration (such as Mars exploration and Asteroid exploration), we can utilize the SPL to rapidly construct corresponding

Integrated design environment system in order to support the development work of new models.

3.1 The Integrated Design Environment of The Luna Probe

When constructing the integrated design environment of the Luna Probe based

on the 2nd Phase of the Luna Project, we should pay attention to the specialties and the

design conditions required by these specialties during the process of designing the

lander and the inspector apparatus, while combining with the construction of the

System Design & Simulation Lab for the Institution of China Aerospace Technology

Research, share the common resources of general aircraft design, and construct the

platform of general design and simulation experiment for the detector system in the

2nd Phase of the Luna Exploration Project.

3.1.1 System Framework

The integrated design environment of the Luna Probe organizes the design tasks

7

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 14

by rules that mainly use collaborated design flow management, controls and manages

the design tools, design data and data flow direction of every design task by the mean

of collaborated design. It also assists in configuring related task analysis software,

math simulation software, parameterized math models and base data, formalizes the

design process and design data, supports task analysis and system design of general scheme, and offers the capability of comprehensively comparing multiple schemes

and targets. In the early development stage, various factors should be considered to

solve problems as early as possible, so as to implement top level optimization,

enhance design efficiency and reliability, and lay a strong base for the general design

and development of the lander and the inspector apparatus.

The Luna Probe Integrated Design Environment consists of four parts: (1) the general design system; (2) the mathematics/semi-physics simulation system; (3) the auxiliary support system; (4) the base platform system. The general framework of the system is shown in Figure 3. Project Management

Figure 3 Function Composition of the Integrated Design Environment of the Luna Probe System

3.1.2 THE MAIN FUNCTIONS

（1） The Normalized Design Process and Data Management 8

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 14

In the Luna Probe Integrated Design Environment, normalized design process

management mode should cover every production stage. It organizes the design

process by certain order and rules, normatively configures related task analysis

software, reusable math models and base data, controls and manages the tools, data

and data flow direction of different tasks by the means of collaborated design. The

mode also effectively organizes design resources, validates the matching of interfaces

between various sub-systems, offers the capability of comprehensively comparing

multiple schemes, analyzes and validates the correctness of design schemes, and

supports the design of the general scheme.

（2） Mathematic/Semi-Physics Simulation Experiments

During the planning, prototyping, production, and in-orbit operation stages of the

detector development, we integrate and manage the math models or hardware

equipments by configuring simulation resources. Using the principle

prototype/electrical property components/evaluation parts, we can rapidly construct

the mathematic/semi-physics simulation environment, perform the

mathematic/semi-physics simulation of the lander/inspector apparatus, thus validating

the interface matching and the general performance of the detector system. During the

planning, prototyping and production stages, the mathematic/semi-physics simulation

experiment system can offer validation data that is more realistic. Even in the in-orbit

operation stage, the system can use math models and some evaluation products to

perform semi-physics simulation analysis, in-orbit failure simulation, and ground

validation of failure handling schemes.

（3） Consolidated Resources Management

By the consolidated management of resources such as simulation calculation, data,

network communication, comprehensive control and demo, we can enhance the

efficiency of calculation and communication resources, ensure the unity，integrality，

and safety of data resources, enable the unified management of safety, system

9

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 14

surveillance, and user privilege, advance the efficiency and capability of the Luna

Probe Integrated Design Environment, thus eventually improving the ability of

general design and simulation.

3.2 The Technical Path of Implementation

The technical path of gradual development, which is based on SPL technology, is

to be used for developing the Luna Probe Integrated Design Environment. The

technical path is divided into three stages, as shown in Figure 4.

Figure 4 Steps of Developing The Luna Probe Integration Design Environment

3.2.1 DIMAIN REQUIREMENTS ANALYSIS

The main activities of this stage are to decide the borders of the domain,

determine the subjects of the analysis, identify the possible information source of the

domain analysis and the whole domain process, and construct the domain model.

(1) Based on the analysis of the requirements of several typical systems in the

domain, we emphasize experts in the domain as the leading members for studying

software production technology of the components-architecture model, and formulate

10

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 14

a standard criterion of space software components that is compatible with

international standard.

(2) In the deep space exploration domain, consider factors such as variability of

prospective requirements, technology development, and objective restrictions, we

determine the appropriate domain borders, identify the commonality and the

variability, obtain a set of reusable domain requirements, develop the model of the

domain of deep space exploration, and describe the common requirements of different

integrated design environment systems .

3.2.2 Domain Design and Development

The primary activity of this stage is the development of core assets.

(1) As shown in Figure 5, identify and develop core assets such as architecture/components in the domain based on the model of the deep space exploration domain, so as to obtain Domain Specific Software Architecture (DSSA).

DSSA is a high level design adapted to the requirements of many systems in the deep space exploration domain, and it describes the solution for the requirements expressed by domain model. Since the domain requirements of the domain model has a certain degree of variability, the DSSA should also has variability.

(2) Based on the DSSA of the model of deep space exploration domain, identify and design core assets such as the architecture/components in the domain; utilize the variability control method of SPL to fully describe the commonality of application systems in the deep space exploration domain; based on existing products, develop coarse-grained, reusable and extendable software architecture (i.e. commonality) and components (i.e. variability), which are required by the Integrated Design

Environment of the Deep Space Exploration Domain (DSED).

11

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 12 of 14

Figure 5 DSSA of The Luna Probe Integrated Design Environment based on SPL

3.2.3 DOMAIN IMPLEMENTATION

The primary activities of this stage are to rapidly construct or improve core assets according to new mission requirements, in order to form new integrated design environment, while keep the evolvement of the core assets.

(1) Define the mechanism that builds application systems by reflecting requirements to core assets. When developing a new application system in the deep space exploration domain according to a new deep space exploration mission, we utilize the deep space domain models and DSSA to develop and organize reusable information, determine the requirement rules of the new application, then choose an

appropriate system framework, and use this system framework as the base for

choosing and assembling components, thus finally form a new integrated design

environment system for the Probe.

(2) Gradually develop components of product line by the way of evolving

existing components, and gradually improve the products in legacy systems; support

professionalized components production, which means developing components

12

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 13 of 14

purposefully and obtaining components from existing systems by applying

re-engineering techniques.

(3) Control the variability and traceability of the domain, and keep the

evolvement and management of the core assets library.

4 Conclusion

Nowadays, our Luna exploration project is taken in the round, and it faces

several layers of pressure such as the increasing number of tasks in pre-research and

model development, the intense project schedule and the pressing demands for the

developments of related supporting systems. As a result, it is essential to introduce

new technologies -- developing the integrated design environment of the deep space

exploration domain. Through the SPL technology, the development of integrated

design environment of the deep space exploration domain can fully utilize existing

development results, in order to rapidly construct new systems or improve upon

existing systems to satisfy new mission requirements, hence eliminating the repeated

work in analysis, design, coding, and testing, etc, and improving the development

efficiency of support systems in the deep space exploration domain. Meanwhile, by

reusing existing high quality products, it is possible to avoid mistakes that might be

introduced by repeated development, hence advancing the quality and reducing the

risk of software products.

5 References

[1] J. Bosch. Product-Line Architectures in Industry: A Case Study. The 21st International

Conference on software Engineering, November 1998.

[2] J. Bosch, Evolution and Composition of Reusable Assets in Product-Line Architectures: A

Case Study, in Proceedings of the First Working IFIP Conference on Software Architecture,

February 1999.

[3] M. Svahnberg, J. Bosch, Characterizing Evolution in Product Line Architectures, in

13

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 14 of 14

Proceedings of the 3rd annual IASTED International Conference on Software Engineering and

Applications, IASTED/Acta Press, Anaheim, CA, 1999. 92-97.

[4] J. Bosch, Design & Use of Software Architectures - Adopting and Evolving a Product Line

Approach, Addison-Wesley, ISBN 020167494-7, 2000.

[5]Bass L, Clements P, Cohen S, Northrop L, Withey J. Product Line Practice Workshop Report

(CMU/SEI-97-TR-003). PA: Software Engineering Institute, Carnegie Mellon University, June

1997.

[6] Bergey J, Clements P, Cohen S, Donohoe P, Krut B, Northrop L, Tilley S, Smith D, Withey J.

DoD Product Line Practice Workshop Report (CMU/SEI-98-TR-007). PA: Software Engineering

Institute, Carnegie Mellon University, 1998.

[7] Atkinson, C., Bunse, C., Jungmayr, S. and Kamsties, E. Systematic Object-Oriented

Development. In proceedings of the GI-workshop “Anwendung von objektorierten

Entwicklungsstrategien undderen Unterstützung durch vorgehensmodelle”, Frankfurt, Germany,

1998.

[8] Atkinson, C., Bayer, J.and Muthig, D.Compent-Based Product Line Development: The KobrA

Approach. 1st International Software Product Line Conference, Pittsburgh, August 2000.

[9] M. Mattsson, J. Bosch. Framework Composition: Problems, Causes and Solutions.

Proceedings TOOLS USA’97, 1997.

[10] Lisa Brownsword, Paul Clements, “A Case Study in Successful Product Line Development”,

Technical Report, CMU/SEI-96-TR-016, October 1996

[11] Sholom Cohen, Seymour Friedman, Lorraine Martin, Tom Royer, Nancy Solderitsch, Robert

Webster, “Concept of Operations for the ESC Product Line Approach”, Technical Report,

CMU/SEI-96-TR-018, September 1996.

14

Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 7

Development of Aerospace System Engineer Technology

Zheng Jinjun, Lin Yiming, Chu Haibin （China Academy of Space Technology, Beijing 100094） Copyright © 2009 by Zheng Jinjun, Lin Yiming amd Chu Haibin. Published and used by APCOSE with permission.

ABSTRACT: Using the system engineering technology to solve the problem in aerospace developing project is important. The paper concludes three phases of spacecraft system engineering development through analyzing development course of international spacecraft system engineering. The architecture, process control and simulation technology of international spacecraft system engineering are introduced. By comparing of domestic development course with international, some illuminations are given. At last, the paper gives some proposals for China spacecraft system engineering development.

Key words: spacecraft system engineering system design and simulation

1．INTRODUCTION

System engineer is belonging to engineer technology class in the system science. As a new subject, system engineer applies to all fields and get the good effects. The practice action of system engineer speeds the self development and becomes perfect. But in order to solve engineer problem, system engineer need to synthesize differ subject knowledge. For person at work on different technology trade, the understandability of system engineer is different. Therefore, the theoretic and method of aerospace system engineer required to get more perfect.

According to aerospace characteristic, aerospace system engineer can be defined as organizing and managing technology for aerospace system layout, research, design, manufacture and test using the theoretic and method of system science. Depending on theoretic model, mathematics simulation, expert agent, economic analysis and experiment improvement, aerospace system engineer harmonizes the system goal, technology and economic feasibility.

Aerospace system product mainly involves launcher, artificial satellite, man spacecraft and missile system. How to make the requirement evolve to aerospace system product is the function of system engineer technology. Aerospace system must consider all-life period action that is from concept research to deploy using, and integrate multi-science and multi-technology, and manage thousands upon thousands persons to coordinate work in many decades, and keep technology, finance and schedule development in balance. System engineer method is one and only choice to organize aerospace system product development and manufacture work[1].

2．AEROSPACE SYSTEM ENGINEER DEVELOPMENT

Origin concept of system engineer had appeared in the long before. For example, KongZi, Chinese thinker, takes nation as unification. But modern system engineer began from 1940s’. Its development can be divided into three phases[1].

1 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 7 (1) Origin phase (from 1940s’ to 1950s’): In 1957, the publishment of “system engineer” written by American scholar H.H.Goode and R.E.Machol means that system engineer becomes new subject. (2) Development phase (from 1960s’ to 1970s’): In 1965, the publishment of “system engineer handbook” written by R.E.Machol means that system engineer turns into primary mature period. (3) Mature phase (from 1970s’): In 1990, the publishment of “system engineer guide” written by System management college of American DOD means that the usage of system engineer had become standardization.

2.1．Foreign spacecraft systems engineering technology development 2.1.1．Origin phase (from 1940s’ to 1950s’)

The first research report “primary design of experiment round-the-world areocraft” provided by the RAND Corporation born in 1948 is the typical system engineer problem. Therefore, the RAND system analysis method that brings immense influence to development strategy of military satellite and ICBM comes into being. In 1940s’, American “Manhattan” plan gives a good demonstration of using system engineer. At the same time, the research work of rocket engine and corporal missile promote aerospace system engineer development. Researcher that is engaged in aerospace system engineer project summarize many experience in project layout, design, experiment and management: (1) Aerospace system must have a system design department to make overall plans and take all factors into consideration. (2) System design department must be given clear responsibilities and sufficient authority, fully ensure system design department and the sub-systems engineering department of the smooth information channels (3) JPL created a matrix structure, that is, by profession to build a matrix model.

In systems engineering theory, Analysis methods have emerged, on behalf of RAND. Its purpose is to provide a major study on the development plan and the corresponding scientific basis, provide a variety of programs and gives the evaluation, and provide analysis of complex problems and solutions.

2.1.2．Development phase (from 1960s’ to 1970s’)

1) "Polaris" missile plan In 1957, the U.S. Navy carried out "Polaris" missiles, including Missiles, submarines, and underwater communication equipment and navigation systems etc. "Polaris" missiles system is very large and complex, and there are many management problems, therefore program evaluation and review technique PERT and critical path method CPM are widely used. Draw a plan network In accordance with the logical order. By calculating the cost and progress, identify the critical path that is composite of Key activities and events throughout the development process. By the adjustment of the key activities and events, the entire project can be completed on time quality and quantity. Using these methods, "Polaris" missile development cycle is shortened by 20% -25%,

2 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 7 two years ahead of schedule to complete the task.

2) "Apollo" moon-landing plan in the 1960s’, "Apollo" moon-landing plan send people to the moon. For the effective implementation of this project, they used a systems engineering approach, create a series of new methods of system analysis and system management against the shortage of PERT, including PATTERN, GERT and VERT etc. "Apollo" moon-landing plan set a good example of spacecraft systems engineering in the following respects: z development and selection of Lunar landing program z developing a detailed and clear development planning z Firmly grasp the research on the environment z Control of the overall performance index z Solve the system interface

In systems engineering theory, there has been represented by a Hall of hard systems engineering methodology., Three-dimensional structure of the Hall system engineering divided the systems engineering activities into 7 stages and 7 steps that are closely linked, and provided a unified way of thinking to solve large-scale complex systems.

2.1.2．Mature phase (from 1970s’)

Since the 1970s’, many countries have introduced a systems engineering, Such as the former Soviet Union began to study engineering from the "control theory", and succeed in the enterprise automation, space technology and national economic planning. Into the 1980s’, "Synergetics", "catastrophe theory" and "theory of dissipative structure" appeared, Reliability engineering, forecasting technology, computer technology, information technology, and fuzzy mathematics were rapid developed and applied, which push the modern systems engineering methods to a new stage.

NASA, ESA, Russia and other countries gradually developed a more complete spacecraft system engineering standards and norms in the development process of the spacecraft for several decades. In 1969, The U.S. military established the U.S. military systems engineering standard Mil-Std-499. In 1998, league of American National Standards (ANSI), Electronic Industries Association (EIA) and the International Electrical and Electronics Engineers Standard team issued the systems engineering standard ANSI/EIA-632. In 2002, International Organization for Standardization established the ISO / IEC 15288 international standards. In 2004, International Advisory Committee Systems Engineering (INCOSE) promulgated the INCOSE-TP-2003-016-02 System Engineering Handbook[2~4].

(1) Spacecraft systems engineering system Systems engineering system mainly includes the following elements[5]: z Common technical processes z Tools and methods z Workforce

3 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 7

Fig1 System engineering frame (2) Spacecraft systems engineering technical process Systems engineering process is as follows.

Fig2 NASA system engineering processing (3) Spacecraft System Engineering Simulation The aerospace sector and the aerospace companies of United States, Russia and other European countries have generally developed and applied the digital spacecraft system design and simulation platform. Several typical spacecraft system design and simulation platform is shown in the table below.

4 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 7

Table1 Spacecraft system design and simulation platform Simulation platform Development agencies Functions AEE(Advanced Engineering NASA Mainly used in reusable launch Environment） vehicle design and analysis SSDSE(Spacecraft System Design NASA The satellite-aided design system, & Simulation Environment） size identification, simulation and cost estimation. MUSSAT(Modeling and Aviation Institute of Modeling the different parts of the satellite, and a wide range of Simulation of Satellite Systems) the Technical options to compare University of Munich EuroSim Federation of The entire process of aircraft European Simulation design (from feasibility analysis to Societies the whole process of experimental verification) of the General Simulation VirtualSat Pro VirtualSat Pro Ltd Virtual environment in a pure or semi-physical simulation environment can be simplified to the space vehicle testing and research. Team-X NASA JPL Widely used in spacecraft program stage, to improve the design of the IMDC GSFC spacecraft and shorten the design CIEL The Boeing Company cycle has played a very important system design role. ICEC UTC and simulation center CDF ESTEC

2.2．Domestic spacecraft systems engineering technology development Systems engineering of domestic space systems technology development is still in development stage. In 1956, the Chinese Academy of Sciences established the first operations research group by Professor Qian Xuesen and Professor Xu Guozhi. In 1958, the publication of the Chinese version "engineering control theory," by Professor Qian Xuesen has laid a theoretical foundation of systems engineering. 1960s’, the famous mathematician Hua Luogeng promoted the co-ordination and optimization method. At the same time, under the leadership of Qian Xuesen, modern weapons such as missiles, space mission design organization has made a wealth of experience. In 1978, Qian Xuesen, Xu Guozhi, Wang Shouyun published the article "organization and management of technology - Systems Engineering", which opened a large-scale development of systems engineering in China. In 1980, Systems Engineering Society of China was formally established in Beijing By 21 experts and scholars, including Qian Xuesen, Song Jian, Guan Zhaozhi, Xu Guozhi[6][7]. China also had an outstanding contribution on system engineering methodology study. In 1989, Qian Xuesen Raised Meta-Synthesis. In 1992, Qian Xuesen Raised Hall for Work Shop Meta-Synthetic Engineering (HWSME). In the end of the 1990s’, Gu Jifa and Zhu Zhichang

5 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 7 raised the WSR System Methodology.

In accordance with system engineering theory and method, Chinese space engineering established scientific rational and efficient organization management systems, applied scientific progress of the project management, quality management and risk management, which played an important role in achieving the sustainable development of space industry, reflected in the following two aspects: (1) Organization and management system provide sustainable development of aerospace industry in the organization aspect. China aerospace science and technology established a scientific and rational and effective organization, command and management system under the guidance of the systems engineering theoretical thinking. (2) Matrix management provide sustainable development of aerospace industry in the technique aspect. Systems engineering for the development of China's aerospace industry has played an important role. At the same time, theories and methods of systems engineering models are developed in practice space has been further enriched and developed. However, in comparison with foreign systems engineering technology, there is still a gap in China system engineering technology, and need further research and development.

3 ． Revelation and recommendations on spacecraft systems engineering technology development 3.1．R Revelation on spacecraft systems engineering technology development (1) Revelation of foreign spacecraft systems engineering technology development: z The improvement of systems engineering system is getting perfect. z The capacity of support in Systems engineering is increasing. z The long-term mechanism of Systems engineering is developing.

(2) Revelation of domestic spacecraft systems engineering technology development: z The development of the process of spacecraft systems engineering technology has not yet formed a complete system of norms. z The experience and application in the spacecraft systems engineering is lake. z Means of spacecraft systems engineering technology is not yet complete.

3.2．Recommendations on spacecraft systems engineering technology development (1) Deep analyze and research on the Laws and institutional of spacecraft systems engineering technology by use of systems engineering technology. Aerospace industry must research on systems engineering technology on the base of theory and method: z Analysis of its structure, functions, characteristics, status and behavior z Analysis of the international and domestic political, economic, and technological impact z Analysis of the relevant government departments, the user units, sub-contractor, and supply-side impact z Analysis of system in the internal factors in the ongoing and adaptation process of the

6 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 7 interaction between people and the environment

(2) Learn systems engineering technology to enhance innovative ability Aerospace industry is such a complex system that the environmental adaptability and innovation development must be its most important characteristics. Innovation is the inevitable result of adaptation to the environment and the spirit of system development.

(3) Create a space systems engineering and technology system With Chinese characteristics Development of the national economic construction and national defense construction demand higher level in the aerospace technical, quality and quantity. Improve efficiency and shorten the development cycle; rational use of resources, lower development costs; to meet the performance requirements to ensure that product quality will become increasingly clear the goal of space development of a management. The managers, scientists and scholars of China aerospace area should participate in the research and discussion of aerospace systems engineering approach, and work together to create technology system of the systems engineering to meet the needs of China aerospace technology.

Reference

[1] Guo Baozhu China Aerospace Systems engineering methods and practices [J] Complex Systems and Complexity Science 2004.4 vol 1 No.2 : 16~19 [2] L. Don Woodruff NASA System engineering handbook[R]: Volume 1 – overview and processes 1994 1~28. [3] EUROPEAN COOPERATION FOR SPACE STANDARDIZATION ECSS Space Engineering System Engineeriing[S] ECSS-E-10A : 1996.4 1~10. [4] Cecilia Haskins. INCOSE System Enginerring Handbook-a Guide for System Life Cycle Processes and Activers[R].version 3,2006 1~29 [5] NASA System Engineering Handbook[R].NASA/SP2007-6105,2007 1~18 [6] Wang Liheng China Aerospace Systems Engineering [J] Aerospace Industry Management 2006.10 60~64 [7] Guo Baozhu Discussion on China's space systems engineering [J] China Aerospace 2003 vol 6 5~9

7 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 15

Development Schemes of Spacecraft Systems Engineering Techniques

Yu Houman Hao Wenyu Yuan Jungang Lin Yiming （Beijing Institute of Space System Engineering, Beijing 100094, China） Copyright © 2008 by HU Zhixin and LIN Yinming. Published and used by APCOSE with permission.

Abstract: Development schemes of spacecraft systems engineering techniques are studied in this paper. Firstly, the current and development requirements of our country’s systems engineering techniques are analyzed. Secondly, the systems engineering framework is constructed, which make systems engineering processes and activities as core elements. Then, the development goals of systems engineering techniques are presented, and the construction contents are put forward from six aspects. Finally, the development approaches of systems engineering techniques are discussed. Key words：spacecraft；systems engineering; systems engineering framework; systems engineering elements; systems design

1. Introduction

Systems engineering significantly demonstrate the capability of developing and innovating spacecraft systems. On the Comprehensive Application of spacecraft engineering techniques and the theories and methodologies of systems science， the main task of spacecraft systems engineering is performing requirements analysis, scheme design, manufacturing and final assembly operations and test and verification for spacecraft systems so as to develop optimization system product which satisfies utilization requirements for the system lifecycle.

Systems engineering technique activities embrace 11 processes, including system definition, objective determination, requirements analysis, system scheme design, product manufacture, final assembly and integration, test and verification, product inspection, evaluation and consignment, operation and maintenance and systems treatment. While the spacecraft systems engineering technique process has its own specific characteristics, for the reason that its assignments have high-input, high-risk and high-reliability characteristics.

The past 40 years witnessed China’s tremendous manned space-engineering achievements. In recent years, the establishment of the spacecraft test center and spacecraft on-track operation management center has resulted in amelioration of systems engineering technique processes and improvement of capability and innovation for spacecraft system research and manufacture.

However, systems engineering technique schemes and capability especially as implementation for domestic crucial technological projects in the medium and long term, construction for Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 15 army-civilian space system and international market competition etc., reflect on new challenges for the development of spacecraft systems engineering techniques. So as to accomplish current missions, raise capability for accomplishing future space missions and actively confront the international market competition, the objective and contents for systems engineering techniques have to be scheduled in the long term with expanding the research and construction for systems engineering techniques and propelling the sustainable and rapid development of systems engineering techniques.

First of all, spacecraft systems engineering technique development requirements are analyzed in this paper. Grounded on the foregoing statement, development schemes of spacecraft systems engineering techniques are considered, the complete framework of the development schemes are constructed, the objective and relative contents for systems engineering techniques are proposed and the approaches are discussed. Above all, the main purpose of this paper is to provide some references for the development of spacecraft systems engineering techniques and the improvement of systems engineering capability in China.

2. Spacecraft systems engineering techniques gap between present and development requirements

During the 40 years’ development of spacecraft systems engineering practice were gained plenty of systems engineering theories and practical achievements, proposed some systems engineering techniques and raised systems engineering techniques to a high level. In comparison with big and powerful space requirements, the gap between present and development requirements, especially in aspects of systems engineering processes, systems engineering standards and criteria and systems design capability, is very explicit at present.

2.1 Gap for systems engineering processes

In the light of systems engineering practice, Program for Satellite Research and Manufacture and Project Phasing and Planning for Space Products etc. were composed at the time of spacecraft research and manufacture. Moreover, the process of spacecraft research and manufacture has been improved with the research of the process restructuring and the mode of batch research and production. However, there are still some drawbacks for constitution of process: 1) the partition and requirements of missions are not definite in the extendability phase of systems engineering; 2) the process is not in detail, usually lacking in the substantially instructive second and third level process for the concrete mission or activity; 3) the definite work process is not instituted and the exploration of the critical activity is not intensive, especially lacking in systematic and in-depth studies of the activities for mission analysis and systems reliability and security design and testing; 4) studies and application of on-track data are not intensified and no effective support is provided for analysis model updating and product design improving.

With contrast to the presently requirement, integrated and clear procedure of systems engineering and the severe control operations should be established consummately, which can Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 15 consolidate and improve the procedure of systems engineering as well as optimizing the systems engineering process and identifying and operating the critical activities according to the requirements of maturity model of systems engineering capability as a mean of strengthening the standardization of the technical activities for mission analysis, systems reliability and security design, testing and on-track application etc..

2.2 Gap for systems engineering standards and criteria

Most domestic spacecraft research and manufacture enterprises established systems engineering standards according to their own business domain. With summarization of the engineering practice, CAST has established integral thermal，mechanical，EMC and anti-radiant standards for reliability design and analysis and criteria for environment testing in recent years. But the present spacecraft systems engineering standards are not comprehensive in the whole lifecycle systems engineering critical activities, such as systematic and comprehensive mission analysis at the beginning of the mission and systematic and comprehensive standards for testing and operation management at the end of the mission. Furthermore, the mechanism, complementing and revising systems standards and criteria as soon as possible with development of techniques and accumulation of experience, needs to improve.

The Aeronautics and Space research or manufacture enterprises of any country should establish considerably comprehensive Space Design Guidelines and Technical Standards that suit for the base of his own industry. At present, in charge of the chief engineer also started a technique project by NASA. In this project, the spacecraft standards will be improved by absorbing non-government standards, exploring new standards, sharing national and international standards, complementing the interpretation of failures and application so as to strengthen standards’ practicality. The project also covers establishing a standard issue updating system. The European cooperation for space standardization (ECSS) established systematic, comprehensive and sustainable updating space engineering standard, namely ESA (European Space Agency) Standardization Steering Board, in the light of ISO (International Organization for Standardization), ESA/PSS (ESA Procedures Standards and Specifications) and standards of other space companies.

2.3 Gap for systems design capability

Systems design and simulation platform is a significant facility of spacecraft integration design and an important approach of improving spacecraft systems design capability. Supported by China’s relative departments, multi-type oriented systems design, testing and on-track support systems were developed. These systems mainly provide service for systems design and simulation. Though the systems possess illustration function, they are not be widely applied on practical projects from preliminary designing phase to the orbit maintenance phase and not unificationly provide support for the entire systems integration design and simulation. Therefore, the cooperation design of multi-type spacecrafts, especially the cooperation design of the preliminary schemes, are not be applied to every domain efficiently and widely. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 15

NASA and ESA etc. explored and applied spacecraft systems design and simulation platform, established cooperation design departments, achieved all-purpose simulation of spacecraft fast design and the entire spacecraft design process (from feasibility analysis to testing and verification) and provided comprehensive support for systems design analysis and simulation of various complex and large space missions. That, the Advanced Engineering Environment (AEE) explored by NASA and the Spacecraft Simulation (SPASIM), are typical spacecraft systems design and simulation platform. While, the Team X in NASA, Concurrent Design Facility (CDF) in ESA, CDC (CNC Design Consultants) in American Aerospace Corporation and the Concurrent Integrated Engineering Lab (CIEL) in Boeing are all typical spacecraft cooperation design departments.

2.4 Domestic situation and requirements of development missions of

spacecraft systems engineering techniques

Referring to spacecraft development, China confronts various situations, such as fast development of space techniques, rapid increase of space missions, transition from applications missions to business missions, construction of army-civilian space scheme, international market competition and improvement of engineer training mechanism. The situation above poses challenges for domestic spacecraft systems engineering implementation, systems engineering techniques, systems engineering manufacture efficiency and sustainable systems engineering development.

That, fast development of space techniques, causes more requirements for engineering technique capability. The significant technical projects are much more complex and come with the need of the lifecycle systems engineering technique process, standardization of critical technical activities and efficient protection for control mechanism.

That, rapid increase of space missions and expansion of the space domain, causes more requirements for manufacture system construction and systems design and integration capability. So as to provide technical support, not only comprehensive manufacture standards and criteria should be composed, but also advanced systems design and integration methods should be proposed and efficient systems design and simulation facilities should be explored.

That, transition from applications missions to business missions as well as construction and steady operation of the army-civilian space scheme, causes more requirements for spacecraft systems reliability and security design and maintenance of the process. Therefore, the reliability and security design should be developed, the procedure of the reliability and security design should be arranged, and comprehensive and detailed criteria for technical activities should be instituted.

Dramatic competition of the international spacecraft market occurred in the domain of navigation satellite. For expanding international market more extensively, spacecraft quality and reliability have to be predominant, that poses more requirements for systems analysis, design and testing as well as reliability design.

That, fast development of space vocation and existence of more and more younger engineers in Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 15 recent years, causes urgent improvement for the engineer training mechanism. Besides the engineer training mechanism should be explored, comprehensive systems engineering technical schemes should be established so as to make a solid foundation for systems engineer training and growing.

3. Systematic framework of spacecraft systems engineering techniques

The framework of spacecraft systems engineering, as a mean of completely analyzing the critical influencing factor for development of spacecraft systems engineering techniques and then clarifying the development orientation and construction contents, should be studied and established in the light of the disciplines of systems engineering and systems research and manufacture.

The process of systems engineering implementation is that identifying critical activities and institute processes of the activities, consequently beginning the lifecycle systems engineering process supported by systems engineering techniques, standards and criteria as well as facilities and methods etc.. Therefore, the soul of systems engineering schemes is the activities and processes, while, the techniques, standards and criteria as well as facilities and methods provide significant support for the activities. Firstly, the lifecycle systems engineering technique process have to be established with instituting and optimizing successively systems engineering activities, disciplining missions and activities of each phase and clarifying their technical requirements and accomplishment indications. Then, on the basis of identifying critical missions and activities of systems engineering and strengthening the systematic research of the missions and activities, clarifying their critical design, analysis and testing, handling their disciplines and substance according to requirements of characteristics and functions of systems missions, have to be established the disciplines of critical missions and activities of systems engineering, partitioned the activities, instituted their processes, established the second and third technical processes and standardized the activities of systems engineering techniques.

For supporting, indicating and standardizing the lifecycle systems engineering activities, have to be established and completed schemes of systems engineering techniques, standardized criteria and schemes of facilities and methods of systems engineering design and testing, established significant maintenance and support systems. Systems engineering techniques are the professional techniques of preliminary design, reliability and security design, testing and verification. Facilities and methods of systems engineering design and verification are the software facilities and methods which are applied in systems engineering missions, for instance, N2 chart, task tree, multi-discipline optimizing methods, mission analysis and design softwares and systems simulation platforms etc..

For guaranteeing sustainable development of spacecraft systems engineering techniques, sustainable improvement mechanism of systems engineering have to be established. That poses successively summing up experience and innovation practice of spacecraft engineering research and manufacture in China, successively investigating and analyzing development of international advanced spacecraft systems engineering techniques, strengthening analyses of operation status and data of domestic on-track spacecrafts so as to improving and standardizing successively systems Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 15 engineering activities and technical processes, enriching and completing constantly systems engineering techniques, standards and criteria and standardizing facilities and methods.

That, completing research organization of systems engineering and systems engineer training mechanism, provides significant support for development of systems engineering techniques. On the one hand, multi-phase research organization of systems engineering and sustainable research mechanism must be established in order to provide support for research and development of systems engineering techniques; on the other hand, systems engineer training mechanism must be completed, mechanism of fostering engineers’ capability, combining with training mechanism and engineering practice, must be explored in order to provide adequate systems engineers. Furthermore, comprehensive and effective mechanism of development of systems engineering techniques must be studied and established so as to improve sustainable research and development of systems engineering techniques.

In summary, the comprehensive framework of spacecraft systems engineering techniques was established (shown as Fig. 1). That indicates future development and construction of China’s spacecraft systems engineering techniques. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 15

Basis of Systems Engineering Systems Realization Process Systems Engineering Development Capability Technical Inheritance of Manufacture & Integration Support Experience Summarization Experience Techniques Diagnosis & Testing Process Summarization Design, analysis & Evaluation Failure Summarization Launching & Oeparation Reliability & Security Design Methods Extraction Environment Design & Verification Integration & synthetic testing Bottleneck Improvement …… Techniques Consolidation Syst ems Desi gn Process Facilities Reference of Requirements Analysis Facilities Design Tools Investigation Overseas achievements Simulation Testing Platform Requirement Definition Organization Data Integration Mechanism Cooperation Mechanism Research Modes Preliminary Design Testing & Verification Methods Research Process Analysis Models …… Methods and Facilities Processes & Activities of Product Systems Engineering Project Bassis Standards & Criteria improvement Work Processes On-track Systems Analysis Develop- Organiza- Interface Techniques Engineers ment tion Methods of Design Performance Evaluation Simulation Testing Completing Failure Analysis Completing Engineer Professional Techniques Organization Training Mechanism Data Research …… Test & Data updating St r at egy Resear ch Gr oup Experience Application Techni que Eval uat i on Prod. Improvement Techni ques Resear ch Gr oup Tr ai ni ng & Pr act i ce Resear ch Pr oj ocet

Responsi bi l i ty of Each Mechani sm of Tr ai ni ng Techni cal Exchange Gr oup

Support of Systems Engineering Development

Fig 1 Framework of Spacecraft Systems Engineering Techniques Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 15

4. Objective of development of spacecraft systems engineering techniques

With regard to the framework of systems engineering, requirements of current and future space missions and the bottlenecks of development of systems engineering in China, the main task of development of systems engineering is establishing comprehensive and in-depth systems engineering processes, methodologies, standards and facilities as a mean of establishing comprehensive and advanced schemes of spacecraft systems engineering techniques, guaranteeing activities of systems design, analysis and testing and achieving advanced technique capability of spacecraft systems engineering.

The concrete objective of development of spacecraft systems engineering is shown as below.

1) For systems engineering processes, will be studied systems top-level design and lifecycle systems activities, clarifying and improving systems engineering processes, strengthen research of critical activities, proposing comprehensive and updating requirements of systems engineering processes, serving as guidelines for development of systems engineering processes.

2) For systems engineering methods, will be completed elements of systems design, analysis and testing, completed facilities and methods of systems design, analysis and testing, establishing comprehensive elements and methodological matrices of systems design, analysis and testing, completely supported activities of systems engineering, established and completed analysis and application of spacecraft on-track data and effectively provided guidelines for consequent improvement of spacecrafts.

3) For systems engineering standards, will be strengthened research of standard strategies of spacecraft systems engineering, improved research of standards and criteria, established lifecycle, well-ordered and advanced systems engineering standards and criteria, and effectively .provided guidelines for activities of systems engineering processes.

4) For systems engineering facilities, will be strengthened establishment of spacecraft systems design and simulation labs, established lifecycle systems design and simulation platforms, completely supported spacecraft cooperation design, simulation verification, analogue testing and on-track mission programming and simulation analysis, achieved integration of data and information with all phases and improved great-leap-forward development of preliminary design. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 9 of 15

5. Contents of development of spacecraft systems engineering techniques

In the light of the objective of development of systems engineering techniques and the framework of systems engineering, the comprehensive systems engineering techniques will be established in aspects of systems engineering processes, technical support systems, development foundation and mechanism, thereby improving technique capability of systems engineering.

5.1 Innovating systems engineering theoretics and upgrading systems

engineering techniques

Studies of innovation of systems engineering theoretics and synthesis of systems design techniques will be strengthened as a mean of providing theoretical and technical support for improving systems engineering capability.

1) Studies of innovation of systems engineering theoretics will be initiated, new modes of research and manufacture will be explored and cost-effectiveness will be increased.

2) Relative systems engineering techniques, especially systems mission analysis techniques, systems scheme design techniques, reliability and security design techniques, analysis and simulation techniques, systems integrated and synthesis testing techniques and synthetically electronic techniques etc., will be enhanced.

5.2 Summarizing practice experience, investigating the development

overseas and emphasizing the application of on-track data

For making a solid foundation of development of systems engineering techniques and improvement of systems engineering capability, will be successively summarized experience of domestic spacecraft systems engineering, investigated international systems engineering techniques and analyzed and applied on-track data.

1) Failures and experience of the 40 years’ spacecraft systems engineering research and manufacture will be summarized extensively so as to refine methodologies and disciplines and clarify bottlenecks of systems engineering; sustainable mechanism of summarization of failures and experience will be established so as to sum up technical processes, critical activities, professional techniques and facilities and methods of systems engineering, consolidate practice experience and achievement of the development, improve bottlenecks and enhance systems engineering capability. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 10 of 15

2) Mechanism of investigating systems engineering techniques overseas, mainly about investigating research and manufacture organizations, modes, processes and facilities and methods as well as technical research, achievement of application and tendency of development, will be established as a mean of guidelines of development of spacecraft systems engineering in China.

3) On-track data of spacecrafts will be collected and applied extensively, systems of on-track data management and application will be established, on-track problems and abnormal phenomena will be analyzed intensively and comparison between spacecraft ground design results and on-track flight results as well as ground testing data and real flight data, will be studied as a mean of providing effective support for analysis model updating and improvement of design schemes, and completing schemes of systems engineering techniques.

5.3 Completing systems engineering processing and clarifying systems

engineering activity procedure

Activities and processes of systems engineering are the kernel elements of the framework of spacecraft systems engineering and the principal part of systems engineering capability. Exploring and optimizing systems engineering processes and identifying and studying of critical systems engineering critical activities are significant sections of establishment of systems engineering techniques.

1) Processes of research and manufacture will be optimized, missions and requirements of each phase will be clarified further, severe process control operations will be constituted, the lifecycle activities of systems engineering will be planned intensively and comprehensive and detailed processes of systems engineering techniques will be established.

2) Each technical activity of systems engineering processes will be clarified comprehensively, critical activities of techniques will be identified, disciplines of activities will be studied intensively, inner processes, input conditions, output product and task requirements of each activity will be clarified and processes of systems engineering activities will be standardized.

3) Measurement of tailoring systems engineering processes for different domains and maturity projects will be instituted.

4) Quantitative description of systems engineering processes will be studied, processes of systems engineering techniques and mechanism of sustained improvement of systems engineering activities will be established. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 11 of 15

5.4 Completing systems engineering methods and ameliorating

systems engineering instruments

Methodologies and facilities of systems engineering are significant bases of systems engineering activities and play an important role in improving systems engineering capability. With clarifying systems engineering activities, methodologies applied in each mission activity will be comprehensively analyzed, studied and improved and schemes of systems engineering methodologies will be completed; with integrating current systems engineering methodologies, facilities, standards and softwares and synthesizing systems engineering activities, facility softwares, expert knowledge and engineering data, data, interfaces, mechanism and environment will be highly integrated and shared and systems design and simulation platforms for the entire lifecycle of spacecrafts will be established.

1) With clarifying systems engineering processes, critical missions will be identified. Furthermore, the missions’ design, analysis and testing elements, especially the elements which correlate with reliability and security, will be completed with in-depth studies.

2) Studies and application of systems engineering methodologies in each phase will be initiated and schemes of systems engineering methodologies will be completed.

3) Systems engineering softwares will be clarified, softwares of systems analysis, design and verification will be explored or introduced, and schemes of systems engineering softwares will be completed.

4) Systems design and simulation platforms will be established. The main task is shown as below.

z Systems design and preliminary design of simulation platforms will be initiated and functions and constitution of the platforms will be clarified.

z Design of integrated framework will be explored.

z Studies of models of systems engineering processes will be initiated on the basis of studies of clarification of systems engineering processes and correlation of systems engineering activities.

z Studies of models and methodologies of multi-discipline design optimization for spacecrafts will be initiated, thereby providing technical support for exploring platform functions of multi-discipline coordinated optimization.

z That, techniques of constructing groups of multi-discipline product development, techniques of reforming processes of product development, techniques of product Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 12 of 15

data management (PDM) supporting concurrent design, techniques of concurrent engineering (CE) oriented computer aided design and management, techniques of computer supported cooperative work (CSCW), will be studied. Problems of critical techniques of systems coordinated design and exploration of simulation platforms will be resolved.

5.5 Completing systems engineering technique standards and criteria

and establishing cultivation mechanism of systems engineering

techniques

1) Establishing and completing systems engineering standards

Arrangement and establishment of standards of spacecraft systems engineering will be promoted, implementation references will be provided for system engineering activities, and capability of standardizing processes of spacecraft systems engineering techniques.

z In light of classification of systems engineering phases and majors, will be designed the framework of systems engineering standards and criteria, and instituted the plan of construction of standards and criteria.

z Current systems engineering standards and criteria, especially those of systems requirement analysis, systems design, systems on-track operation management and reliability and security, which are relative weak, will be comprehensively clarified, complemented and completed.

z Studies of standards of interfaces among large systems will be promoted.

z With development and practice of spacecraft systems engineering, compilation of spacecraft systems engineering handbooks will be developed steadily so as to guide comprehensively systems engineering processes and provide teaching material for systems engineering training.

2) Establishing training mechanism of systems engineering techniques

With requirements of domain development, will be clarified mature techniques, on-research techniques and demonstration techniques, established technical schemes combining with elements of majors, domains and maturity, and achieved cultivation and arrangement of our own pillar techniques、pillar product and pillar industry so as to provide guidelines for technique development and project application etc., and guarantee close-knit combination between professional technique development and requirements of systems capability as well as that between infrastructure construction and requirements of systems capability. Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 13 of 15

5.6 Completing the organization of systems engineering, constituting

systems engineer training mechanism and perfecting development

mechanism

The research organization of systems engineering will be completed, and systems engineering training mechanism will be completed, thereby guaranteeing constant studies and development of systems engineering techniques.

1) Organizations of systems engineering (organization of engineers)will be optimized, operation mechanism of systems engineering will be completed, and systems engineering processes will be developed orderly and effectively; multi-stage research organizations of systems engineering (systems engineering strategy research group or systems engineering technique research group) will be established, functions of each research organization will be clarified and sustainable studies of systems engineering will be guaranteed.

2) The systems engineer training platform will be established with instituting practice programs, exploring and establishing cultivation mechanism and evaluation standards of engineers’ capability according to the combination between training and practice, thereby enhancing steadily engineers’ capability in systems engineering techniques.

3) Development mechanism of systems engineering will be completed and good circumstances of technical research and exchange will be exerted.

6. Development approaches of spacecraft systems engineering techniques

The implementation of development of spacecraft engineering techniques is shown as below.

1) In the light of the current situation and requirements of future space missions, will be demonstrated the program of spacecraft systems engineering development, instituted definite and feasible development schemes and executed steadily construction of systems engineering techniques.

2) Research organizations of systems engineering techniques will be established, and constant development and research mechanism will be established, thereby propelling and guaranteeing constant studies of systems engineering techniques and improvement of systems engineering capability.

3) Projects must be developed with organic organizations and effective work mechanism. For Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 14 of 15 the entire personnel is the principal part of systems engineering construction, responsibilities of staffs in each phase, will be clarified, thereby achieving close-knit combination and mutual benefit between technique research and product manufacture, guaranteeing that studies of systems engineering techniques propel the improvement of product research and manufacture.

4) During the 40 years’ development, plentiful systems engineering experience was achieved in China, meanwhile, abundant achievements of research and application were gained. Therefore, schemes of systems engineering techniques will be developed on the basis of summing up intensively the experience and absorbing sufficiently the achievements.

5) Cooperation in China will be developed extensively, common construction and achievement share will be achieved, systems engineering development will be enhanced and systems engineering capability in China will be raised comprehensively.

7. Conclusion

Systems engineering techniques are important synthesis techniques for spacecraft research and manufacture. Spacecraft systems engineering techniques are substantially significant for guaranteeing missions’ success and enhancing research capability. In the light of requirements of systems engineering techniques for current and future space missions and bottlenecks in the development of systems engineering, domestic departments of spacecraft research and manufacture should institute a long term program of systems engineering development, establish and complete research organization and mechanism of systems engineering techniques, steadily complete schemes of systems engineering techniques, enhance systems design, strengthen systems verification, guarantee arranged and effective implementation of systems engineering processes and activities, comprehensively enhance spacecraft systems engineering capability.

References

[1] Tsien Hsue-shen. 论系统工程[M]. 长沙：湖南科学技术大学出版社，1988 [2] Guo Bao-zhu. China Aerospace Systems Engineering Methods and Practices [J]. Complex Systems and Complexity Science, 2004, 1 (2) : 16-19 [3] Cecilia Haskins. INCOSE system engineering handbook-a guide for system life cycle processes and actives [M]. Version 3. INCOSE, 2006 [4] NASA Headquarters. NASA system engineering handbook[M]. NASA/SP2007-6105. Washington: National Aeronautics and Space Administration, 2007 [5] Space & Missile Systems Center. SMC Systems Engineering Primer & Handbook [M]. Version 3. Air Force, 2005 [6] Yuan Jia-jun. http://10.66.196.4/web/BsaeView.aspx?ID=930411神舟飞船系统工程管理[M]. 北京：机械工业出版社，2006 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 15 of 15

[7] Wang Li-heng. System Engineering in China Aerospace Industry [J]. Aerospace Industry Management, 2006, 10: 60-64 [8] Yu Hou-man, Fan Han-lin. Achievements and Prospect of Spacecraft System Design Technology [J]. Spacecraft Engineering, 2008, 17 (4) : 1-5 [9] NASA Headquarters. NASA Procedural Requirements[R]. NPR 7123.1A. Washington, US: National Aeronautics and Space Administration, 2007 [10] EIA. System engineering capability model[R]. European Space Agency, EIA/IS-731.1, 1997 [11] ESA Standardization Steering Board (ESSB). ESA Approved Standards[R]. European Space Agency, 2003 [12] Wanda M. Austin. Concurrent engineering of space systems[C]. Brazil: International Society for Productivity Enhancement, 2007 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 1 of 8 The Integrated System for Spacecraft Simulation and test SUN Ya-Nan TU Xin-Ying XIANG Kai-Heng LI Zhi China Academy of Space Technology (CAST) Copyright © 2009 by SUN Ya-Nan, TU Xin-Ying, XIANG Kai-Heng and LI Zhi. Published and used by APCOSE with permission.

Abstract: An integrated system for spacecraft simulation and test which is applicable to the satellite development process of CAST（China Academy of Space Technology） is established in this article. Based on the definition and analysis of its function, a flexible system is put forward. The system includes 3 modules: FE（Front-End） equipment, which connects the tested system to get the test parameters; SE（Simulation Environment）, which can simulate all the spacecraft subsystem and space environment factors, spacecraft dynamics, spacecraft orbit; CE（Checkout Environment）,which executes the test and simulation sequence automatically, and control the entire system. Each module is then described in detail and the interfaces between different modules are defined. In the last section of the article, different system configurations which is adapted to different applications such as EGSE(Electrical Ground Support Equipment) system evaluation and validation, spacecraft gradual incremental test and research of spacecraft failure handling during different development phases are put forward. The integrated system not only support the test in all spacecraft development phases, from spacecraft conceptual design, detailed design and spacecraft manufacturing to AIT, but also can be used in spacecraft failure analysis. The spacecraft development system engineering process will be optimized by the integrated system, and finally the spacecraft development efficiency will be increased. Key words：spacecraft; simulation; test; Integrated

1 Introduction Testing technology and simulation technology as an inspection, verification and design tools which are indispensable during spacecraft development systems engineering play an important role in the process of spacecraft development. Like with any project, spacecraft in the development process from beginning to end can not be separated from a large number of test works. In order to verify the performance and functionality to satisfy the design requirements, in order to get the qualitative and quantitative parameters for the processing and evaluation, we all require to test the spacecraft. Test data and results of the analysis offer the basis to study and improve design[1,2]. Simulation also is an important means to support the development of spacecraft. By the use of the controllability and safety, non-destructive, not subject to environmental conditions and repeated several times, etc., we can verify the design of spacecraft and evaluate the design performance. Simulation results can be used as a reference for improving the design. Because simulation technology also has verified function, so it can be blended with testing technology. Thus it provides the internal demand for the integration of simulation and testing technology. Development of satellite test equipment in the process of commissioning, acceptance, and

1 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 2 of 8 the spacecraft during testing would have to be supported by means of simulation technology. First, when the spacecraft has not yet been built, it is not possible to be test, we can connect the simulation equipment with test equipment in order to verify the function of the EGSE (Electrical Ground Support Equipment) and the test equipments; the second, during the process of spacecraft comprehensive testing, due to some subsystems or equipment not be ready, it is necessary to provide the simulation equipments of these subsystems or equipments, so as to help the testing of the or equipments. In addition, during in the process which spacecraft is integrated test on ground, in order to test the spacecraft flight process realistically, simulation system also needs to constitute the closed-loop signal environment which is required by the spacecraft. In this paper, based on our current spacecraft simulation technology and test technology, a scheme of an integrated system for spacecraft simulation and test is put forward. Through the integrated application of spacecraft simulation and test of technology, we can provide a more efficient means of spacecraft authentication, verification which can improve the parallel degree of the spacecraft development, thus improving the development process of spacecraft systems.

2 Significance and necessity Spacecraft systems engineering is based on scientific theories and methods of systems engineering, integrated light, machine, heat, electricity, communications, management of a wide range of expertise to carry out spacecraft systems needs analysis, program design, manufacture and assembly, validation and use of work, which is the overall optimization system engineering process of developing the system life cycle to meet the application requirements. How to take cost-effective technology to ensure the correctness and effectiveness of verifying the spacecraft is an important element of the study of spacecraft systems engineering[3]. In this paper, the integrated system for spacecraft simulation and test which is put forward by this paper, is the focus on this point raised.

2.1 The position in the spacecraft development systems engineering

Verification is an important aspect of spacecraft development systems engineering，It is directly related to the development of the quality of spacecraft and it relates to the achievement of objectives of spacecraft development systems engineering. In the traditional spacecraft development process, Comprehensive testing on ground is the main means of verification However, in accordance with the existing spacecraft development process, Only when all subsystems of the spacecraft are completed the development and assembly, we can carry out the comprehensive test. In addition, simulation is also a means to verify of spacecraft development system engineering. Unlike comprehensive test, the application of simulation throughout the various stages of spacecraft development, however, the application of simulation technology in the current process of spacecraft development lacks systematic. For example, the common elements of simulation and results can not be shared in various stages. Because of simulation and testing is an important means to verify during spacecraft development process, this provides an inherent demand for the integration application of simulation technology and testing technology. Through the use of the integrated system for spacecraft simulation and test, On the one hand, simulation technology can be used to build test Object, we can advance the spacecraft test stage, to improve the parallel degree of the verification work and other work during the spacecraft development process. On the other hand, the Systematic application of simulation technology during the spacecraft development process will also be improved. At last, the spacecraft design and

2 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 3 of 8 development level will be enhanced, and the existing spacecraft development systems engineering process will also be improved.

2.2 The roles in various stages of spacecraft development

In spacecraft design stage, through the construction of the integrated system for spacecraft simulation and test, simulation the spacecraft and its subsystems by use of simulation technology, we can provide virtual test object for the comprehensive EGSE development, thus supporting the analysis of test tasks and the corresponding test program strategy during the EGSE development, improving the design targeted of EGSE design. In addition, through the simulation modeling of the spacecraft, we can provide intuitive analysis means for the design of interface between sub-systems. And we can also preliminary simulate the interface data and signal characteristics, simulate data and information flow in the spacecraft work process, provide a reference design to help the design of interface between equipments of EGSE. In spacecraft development stage, through the construction of the integrated system for spacecraft simulation and test，we can establish a virtual spacecraft which is the same with the test spacecraft by use the simulation technology, so we can provide a “virtual test validation Environment” for the design and Debugging of EGSE. This will not only able to complete the test of EGSE, but also avoid the damage to the spacecraft which was may cause by use the real spacecraft equipments. In spacecraft AIT stage, through the construction of the integrated system for spacecraft simulation and test, On the one hand, we can provide the operating environment and test means for the spacecraft system or equipments by use simulator. We can support the gradual increment of spacecraft test by the HIL(Hardware-in-the-loop) simulation mean. On the other hand, through the analysis of simulation data, we can also support the interpretation and diagnosis of the spacecraft test data. In the fault diagnosis and processing mean study of spacecraft, through the construction of the integrated system for spacecraft simulation and test, we can simulate the spacecraft fault state for the assessment of the effectiveness of processing the spacecraft fault. At last, through the construction of the integrated system for spacecraft simulation and test, we can train the testers for the EGSE.

3 System Design The integrated system for spacecraft simulation and test (Hereinafter referred to as integrated system) is composed by simulation models and simulator which simulate the spacecraft subsystems and equipments, and spacecraft dynamics simulator, and EGSE, and the interface equipments which exchange signal between the spacecraft equipments and simulation equipment and test equipments.

3.1 Function definition

1）the function of spacecraft scheme design phase： （1） Producing the fasting system prototype by simulation models, constituting a simulation test verification environment, supporting the choice of feasibility scheme and the assessment of design performance; （2） Developing the simulator by HIL technology, providing the electronics signal interface

3 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 4 of 8

which is the same with the spacecraft equipments, build the test verification environment for key technology Prototype test and Assessment （3） Building the simulation verification environment which supports the EGSE development, and the analysis of design demand of EGSE, and the development of test equipment； 2）the function of spacecraft development phase： （1） The function of testability study。Providing virtual spacecraft platform, supporting the spacecraft equipments testability study （2） The verification functions of EGSE development. Providing the test objects by simulation system in order to support the design and debugging of EGSE； （3） The auxiliary function of spacecraft electronics test。Generating the test drive by use of simulation system，supporting the gradual increment of spacecraft test, improving the spacecraft development efficiency； （4） The function of fault diagnosis and processing mean study of spacecraft。Simulating the spacecraft fault, supporting the fault diagnosis and processing mean study； （5） The training function of spacecraft testers。

3.2 The composition of the integrated system

The architecture of integrated system is shown in Figure 1.

Fig.1 Structure of the integrated system Fig.2 Constitution of the integrated system In the application the integrated system can be divided into four levels: 1) The tested system； 2) Front-end interface equipments FE。It is the interface equipments layer which exchanges the signal and data between the checkout environment, simulation environment and the tested system；

4 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 5 of 8

3) Simulation environment SE. To achieve the simulation function, including simulation models, semi-physical simulation simulator resources, such as simulation equipment; 4) Checkout Environment CE. To achieve Simulation test sequences self-executing, controlling the test and simulation process of the integrated system, analyzing test and simulation data。 The composition of the integrated system is shown in Figure 2.

3.2.1 Front-end interface equipments FE 3.2.1.1 Function 1) To obtain the output data or signal of the tested system or equipments； 2) Exchange the signal between with SE and the tested system，supporting the tested system to run and test. 3.2.1.2 Composition The front-end interface equipments FE is composed of the following equipment: 1) The SCOEs(Special Check Out Equipment) ： including electronics power subsystem, communication subsystem, OBDH, GNC, data transmission, and payload SCOE etc.; 2) The simulation signal interface equipment：it can exchange the electronics signal between the SE and the tested system, as the necessary input to the tested system; 3) Signal acquisition monitoring equipment：collecting the test signal and simulation signal which is exchanged by the integrated system and the tested system, for the CE master control equipment to analyze and record; 4) The spacecraft bus interface equipment：To connect with the spacecraft bus, so the bus signal can be collected, and in order to analyze the situation of bus communication. The equipment has 1553B, CAN, RS485 and other spacecraft bus interfaces. 3.2.2 Simulation environment SE 3.2.2.1 Function 1) Realization of spacecraft dynamics, space environment simulation, etc 2) Simulating the spacecraft subsystems and function modules, including mathematical simulation model, real-time simulation software, and semi-physical simulators which have output the same as spacecraft. 3) Control and manage the simulation process； 4) Record the simulation data and stored in the database; 5) Simulation data on-line or off-line analysis and performance assessment； 6) The Demonstrate function。 3.2.2.2 Composition simulation environment SE includes the following node equipment： 1) The simulation Scheduling and management node: control the simulation process, start, stop and pause the simulation; 2) Simulation task design node: According to the simulation needs to complete the design of

5 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 6 of 8

simulation tasks； 3) Analysis of simulation results and performance evaluation node：it can drawn curve based on simulation result data, and assessment the simulation according to the simulation data; 4) Database servers: including simulation result database, simulation model library database; 5) Real-time simulation software: the simulation software package in accordance with a unified interface to manage and expand; 6) General Simulation target machines: As the carrier to run the simulation software of mathematical models; 7) Semi-physical simulation simulators: simulate the spacecraft subsystems or equipments in the form of hardware simulation. The simulator’s output interface is the same with the corresponding spacecraft equipment; 8) The network environment: SE network is constitute of the Ethernet network, the real-time fiber-optic network, and the electronics signal network; 9) Visualization system: real-time display the three-dimensional or two-dimensional spacecraft movement, by the simulation and test data-driven, 3.2.3 Checkout Environment CE 3.2.3.1 Function Checkout environment CE is the control and management center of the integrated process of simulation and test. It provides the function of unified control, management, monitoring and scheduling, during the collaborative process of test and simulation. 1) The implementation of the test sequence and Test process management and control; 2) Management and control the integration systems; 3) Management the test database and spacecraft database; 4) Organizing the fault testing, analysis, diagnosis and treatment of authentication； 5) Collecting the simulation and testing data, and monitoring the process status. 3.2.3.2 Composition

SD

123456789101112

Fig.3 Constitution of CE The composition of CE is shown in Figure 3, and it includes the following node equipment: 1) Master Control Computer: to achieve the initial data preparation, process definition and

6 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 7 of 8

their organizations; to provide the time, data synchronization mechanisms between the EGSE and simulation system; to provide the real-time control for the integrated system; 2) Data display terminal: for the display of test and simulation data; 3) Handle terminal: on-site observation of the operation control and monitoring; 4) Spacecraft database: a record of all the measured spacecraft subsystems and electronic equipment performance parameters, design specifications, as well as the input and output signal characteristics of spacecraft functional modules, to prepare for the inquiry at any time during the test; 5) Test, simulation database: a record of test and simulation data for online and offline analysis, interpretation and playback; 6) FE / SE interface devices: accessing the data from the tested system which were collected by the FE, access the output data of semi-physical simulator; 7) TMFEE: through a wired or wireless channel to obtain the measured spacecraft telemetry information, or the telemetry information which was simulated by the simulation system (at this time for the tested system / environment for debugging, testing and verification); 8) TCFEE: to send remote control commands to the test spacecraft or the "virtual spacecraft" which was simulated by the simulation system, through the wireless or cable channel; 9) The equipment for spacecraft failure test, diagnosis and treatment: to manage and control the activity of spacecraft failure diagnosis and treatment study; 10) Network Environment: all the CE and SE and FE node devices are connected via Ethernet to transfer data and the order. 3.3 System configurations of typical applications

The configuration of the integrated system has a considerable degree of flexibility. The integrated system can be adjusted its architecture and composition according to the application requirements of different spacecraft development stages. The system composition can be adjusted in order to adapt to According to the following needs: 1) For scheme evaluation and performance assessment; 2) For the key technology prototype test; 3) For the spacecraft EGSE development; 4) For satellite sounding progressive incremental; 5) For the spacecraft failure test; 6) For the spacecraft failure analysis, diagnosis and treatment validation; 7) For testability research of spacecraft equipment. Next, the system composition which was used in gradual incremental spacecraft test will be introduced. During the spacecraft AIT phase, the spacecraft electronic equipments will be gradually produced in accordance with the development schedule. After all the spacecraft subsystems and equipments development and assembly were completed, traditional spacecraft test can be carried out. If the integrated system was constructed, we can begin the spacecraft test on ground when only a part of spacecraft subsystems or equipments were manufactured. At this time, we can use simulation SW or

7 Proceedings of the 3rd Asia-Pacific Conference on Systems Engineering (APCOSE), Singapore, 2009 Page 8 of 8 simulator to simulate the spacecraft subsystems or equipments which were not manufactured. The function of those subsystems and equipments can be simulated by simulation SW, and their electronics signal interface can be simulated by the semi-physical simulation interface device. So, we can carry out the comprehensive test to the “virtual spacecraft” which were composed by real spacecraft equipments and simulation SW and HW. With the completion of the equipment being developed, we can use the real equipment to replace the corresponding simulation software simulator or semi-physical simulation at any time. Real equipment and simulation software simulator or simulation achieve the functions of the test spacecraft and provide the "real" signal output. This process continued until all of the equipment has been manufactured, at that time, the spacecraft will be all test. Therefore, through the application of the system, we can advance the spacecraft test stage, to improve the parallel degree of the verification work and other work during the spacecraft development process. And ultimately we can improve the efficiency of the spacecraft development and shorten the development cycle. In such applications, the system configuration including all parts and its composition is shown in Figure 2. 4 Conclusion

Through the use of the integrated system for spacecraft simulation and test, On the one hand, simulation technology can be used to build test Object, we can advance the spacecraft test stage, to improve the parallel degree of the verification work and other work during the spacecraft development process. On the other hand, the Systematic application of simulation technology during the spacecraft development process will also be improved. At last, the spacecraft design and development level will be enhanced, and the existing spacecraft development systems engineering process will also be improved. References: [1]Wang Qingcheng, spacecraft electronics testing technology [M] The first edition beijing Science and Technology of China Press 2007 [2]Zhu Weibao，Wang Qingcheng，Wang Xianwen. Design and Implementation of Spacecraft Overall Checkout Monitoring & Management System [J]. Computer Automated Measurement & Control, 2000.8 [3]Wang Liheng. System Engineering in China Aerospace Industry. AEROSPACE INDUSTRY MANAGEMENT 2006.10

孙亚楠（1978—），男，工程师，从事航天器系统仿真及其应用研究。

8