Evolving Intentions: Support for Modeling and Reasoning About Requirements That Change Over Time

Evolving Intentions: Support for Modeling and Reasoning about Requirements that Change over Time

Alicia M. Grubb

A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy Department of Computer Science University of Toronto

Evolving Intentions: Support for Modeling and Reasoning about Requirements that Change over Time

Alicia M. Grubb Doctor of Philosophy Department of Computer Science University of Toronto 2019

In early-phase requirements engineering, modeling stakeholder goals and intentions helps stakeholders understand the problem context and evaluate tradeoffs, by exploring possible “what if” questions. Prior research allows modelers to make evaluation assignments to desired goals and generate possible selec- tions for task and dependency alternatives, but this treats models as static snapshots, where once the evaluation of the fulfillment of an intention is determined, it remains constant. Using these techniques stakeholders are unable to reason about possible evolutions, leaving questions about project viability unanswered when the fulfillment of goals or availability of components is not guaranteed in the future.

In this dissertation, we propose Evolving Intentions: a framework for specifying, modeling, and reasoning about goals that change over time, enabling stakeholders to explore model evolution. We present Evolving Intentions in the context of both the i* and Tropos goal modeling languages. We specify a set of functions that deﬁne how intentions and relationships evolve, and use simulation strategies for asking a variety of “what if” questions about such changes.

We present GrowingLeaf and BloomingLeaf , two web-based goal modeling and analysis tools that implement this technique for iStar and Tropos models, respectively. Using the development of Grow- ingLeaf as an example, we demonstrate that this technique is eﬀective for debugging goal models and answering stakeholder questions, and show the analysis to be scalable for representative goal models.

We describe a between-subjects controlled experiment that empirically validated the eﬀectiveness of our approach and tool usability. We also report on the applicability and eﬀectiveness, of our technique on a substantial case, where we use historical data and rational reconstruction to understand how a project evolved in the past, and explore alternative futures.

ii Acknowledgements

I feel incredibly fortunate to have been supervised by Marsha Chechik over the past five years. Marsha taught me to think about my thinking and critically examine everything. She helped me confront my demons and compelled me to gain expertise in unexpected areas. Marsha championed my research and was my keeper in difficult times. I sincerely appreciate her unwavering commitment to my academic development and to this dissertation. Thank you Marsha for this transformative experience! I am grateful for all those who contributed to this dissertation. I would like to thank my examination committee for supporting my process, asking the right questions, and giving helpful feedback at every stage: Betty H.C. Cheng, RenéeJ. Miller, John Mylopoulos, and Eric S. Yu. I would also like to thank my collaborators and those who gave feedback on early drafts: Daniel Amyot, Jeffrey S. Castrucci, Fabiano Dalpiaz, Jennifer Horkoff, and Gunter Mussbacher. I was privileged to study in a community of strong scholars who showed passion for both their research and teaching. I am thankful for the personal growth and mentoring opportunities I have had over the years and for my mentors: Joanne M. Atlee, Andrew Begel, Christina Christara, Angela Demke Brown, Sven Dickinson, Steve M. Easterbrook, Faith Ellen, Tom Fairgrieve, Diane Horton, Vijay Ganesh, Sheila McIlraith, Karen Reid, Khai Truong, Todd L. Veldhuizen, Robert Vipond, Andrea Williams, and Mathew Zaleski. Additionally, in completing this dissertation, I felt very supported by fabulous administrative staff: Lynda Barnes, Marina Haloulos, Sarah Lavoie, Joseph Raghubar, Alan Rosenthal, Lloyd Smith, and Ingrid Varga. I would like to thank Michalis Famelis for believing in me and cheering me on at every step. I am grateful to Jocelyn Simmonds for modeling authenticity and integrity. I appreciate Zachary Kincaid for his sincerity and sense of humour in explaining important concepts to me. Federico Mora helped me supervise students and kept things interesting by spawning fruitful debates. L.S. Nick Fung repeatedly came to my aid by copy editing important documents. Yi Li taught me to always finish strong. I am grateful for the six-month-club for keeping me accountable to my goals with our Friday reports, and empathizing whenever I needed support: Emily J. Clare, Clarissa Forbes, Alexandra Motut, and Tomohiro Yokoyama. As part of the Dynamics of Global Change Collaborative Program, I had the pleasure of contemplating wicked problems and learning from those in other disciplines: Keita Demming, Andre R. Erler, Will Greaves, Tim Lang, and Jennie Phillips. I would also like to thank Allison Enright and Christopher Eagle for bouncing ideas around with me about writing in computer science. I am grateful to my many colleagues in the Department of Computer Science. As peers you shared in the ups and downs of graduate school, taught me many life lessons, challenged my wacky ideas and were party to my merriment: Mohammad Ahmadi Danesh, Aws Albarghouthi, George Amvrosiadis, Jorge Aranda, R. Benett Axtell, Zia Babar, Alessio Di Sandro, Cynthia Disenfeld, Golnaz Elahi, Neil Ernst, Hesam Esfahani, Dustin Freeman, Daniel Fryer, Jonathan Eidelman, Phillipa Gill, Bahar Ghadiri Bashardoost, Sahar Kokaly, Aamod Kore, J¯anisL¯ibeks, Jonathan Lung, Mike Maksimov, Lev Naiman, Soroosh Nalchigar, Victor Nicolet, Nan Niu, Peter O’Donovan, Vik Pant, Elizabeth A. Patitsas, Niloofar Razavi, Julia Rubin, Rick Salay, Mahsa Sadi, Ramy Shahin, Ioanna Stavropoulou, Ioan Stefanovici, Anya Tafliovich, Patricia Thaine, and Chenguang Zhu.

iii Erich Welz put the life in work-life balance by ensuring that I stayed grounded, exercised, experienced nature, and explored the city. I am grateful for the encouragement of my friends near and far. Whether late night conversations before deadlines or finding time for tea and checkins, I appreciate how you sustained me: Okhaide Akhigbe, Kaitlin Alexander, Sintia Bejatovic, Annemarie Brissenden, Antonio Carito, Jason Carter, Natalie Crain, Liane F. Daiter, Patricia DeHart, Kathleen Gaudet, Matthew Graf, Megan Grant, Kristopher Gurnsey, Derek Hans, Vega Janic, Fatima Kamenge, Bryan Keith, Anne Keller, Levi McCulloch, Angus McQuarrie, Sabrina Paez-Parent, Oliver Romaniuk, Julian Spillane, Justin Tobia, Neeti Wagle, Kai Wallace, Leah Weller, and Andrew Zamojc. My support network kept me in physical and mental health enabling me to focus on research: Allison Burges, Scott Clark, Susan Harrison, David Lowe, John Molot, Sian Owen, Helen Slade, Becky Smith, and Roderick J. Walker. I acknowledge that this research was hosted on the lands of the Mississaugas of the Anishinaabe, the Haundenosaunee Confederacy and the Wendat, and I want to recognize the enduring presence of all First Nations, Métisand the Inuit peoples. I would like to acknowledge the Department of Computer Science at the University of Toronto for supporting my studies. Bell Canada, Google, Huawei, the Kappa Kappa Gamma Foundation, and the Kappa Kappa Gamma Foundation of Canada provided funding for my research and studies through scholarships, while CRA-W, Microsoft, and SIGSOFT provided travel funding for me to attend conferences and workshops. Finally, I would like to thank my family. I am grateful to my parents, Gerard & Marilyn, for supporting me on this long journey and answering my many inquiries. My grandmother Martina inspired me to seek the highest level of education, I am so glad she stuck around to see me through. Happy 101st Birthday Grandma! Thank you to the rest of my family for giving me space to focus and reminding me of the beauty of the seasons, specifically Alex, Ben, Felix, Isaac, Josh, Laura, Lisa, Martina, Nora, Rita, and last but certainly not least Simon. Thank you!

iv Contents

List of Tables viii

List of Figures ix

List of Algorithms xi

List of Theorems and Deﬁnitions xii

I Motivation & Prior Work1

1 Introduction 2 1.1 Preface...... 2 1.1.1 Behavioural Systems...... 2 1.1.2 Designing for Anticipated Behavioural Changes...... 3 1.1.3 Designing for Anticipated Structural Changes...... 3 1.2 Decision Making in Early Requirements...... 3 1.2.1 Motivating Example I: Waste Management Example (WME) in iStar...... 4 1.2.2 Motivating Example II: Bike Lanes Example (BLE) in Tropos...... 6 1.2.3 Gaps Identiﬁed In This Dissertation...... 7 1.3 Contributions...... 8 1.3.1 Modeling Evolving Intentions...... 9 1.3.2 Reasoning with Evolving Intentions...... 9 1.3.3 Tool Support for Modeling and Reasoning...... 10 1.3.4 Validating Modeling, Reasoning, and Tooling...... 10 1.4 Thesis Organization...... 11

2 Preliminaries 12 2.1 iStar Syntax and Semantics...... 12 2.1.1 iStar Overview By Example...... 13 2.1.2 Model Syntax...... 13 2.1.3 Intention Evaluation...... 15 2.1.4 Relationship Semantics and Propagation...... 15 2.2 Tropos Syntax and Semantics...... 19 2.2.1 Model Syntax...... 19

v 2.2.2 Intention Evaluation...... 20 2.2.3 Relationship Semantics and Propagation...... 21 2.2.4 Propagation Algorithms...... 23 2.2.5 Goal Graph Evaluation...... 25 2.2.6 Evaluation Algorithms...... 26 2.3 Summary...... 33

II Evolving Intentions Framework 34

3 Modeling Evolving Intentions 35 3.1 Modeling in Tropos...... 36 3.1.1 Time Points and Intention Evaluations...... 36 3.1.2 Evolving Functions for Intentions...... 36 3.1.3 Evolving Functions for Relationships...... 40 3.1.4 Specifying Evolving Models...... 42 3.1.5 Completeness...... 44 3.2 Modeling in iStar...... 45 3.3 Summary...... 47

4 Reasoning with Evolving Intentions 49 4.1 Reasoning in Tropos...... 50 4.1.1 Graph Evaluation...... 50 4.1.2 Generating Simulation Paths...... 52 4.1.3 Graph Evaluation Path Algorithms...... 54 4.2 Reasoning in iStar...... 60 4.3 Using Reasoning and Simulation Strategies to Answer Stakeholder Questions...... 61 4.3.1 Simulation Strategies...... 61 4.3.2 Answering Questions in Motivating Example I: Waste Management Example (WME) 63 4.3.3 Answering Questions in Motivating Example II: Bike Lanes Example (BLE).... 66 4.4 Summary and Related Work...... 70

III Tooling & Validation 72

5 Tool Support 73 5.1 GrowingLeaf...... 74 5.1.1 GrowingLeaf Overview...... 74 5.1.2 GrowingLeaf Architecture...... 77 5.1.3 GrowingLeaf Discussion...... 77 5.2 BloomingLeaf...... 78 5.2.1 BloomingLeaf Overview...... 78 5.2.2 BloomingLeaf Architecture...... 82 5.3 Using the Evolving Intentions Framework...... 85 5.4 Summary and Related Work...... 86

vi 6 Validating Evolving Intentions in iStar 87 6.1 Initial Validation: Effectiveness and Scalability...... 88 6.1.1 Effectiveness: GrowingLeaf Analysis...... 88 6.1.2 Scalability...... 90 6.2 Controlled Experiment: Effectiveness and Usability...... 93 6.2.1 Experiment Methodology...... 93 6.2.2 Results...... 97 6.2.3 Discussion...... 105 6.2.4 Related Work: Experiment Methodology...... 107 6.3 Summary...... 107

7 Validating Evolving Intentions in Tropos 109 7.1 Rational Reconstruction Methodology...... 110 7.2 Modeling Scenes...... 112 7.2.1 Scene I (1960–1964)...... 113 7.2.2 Scene II (1970–1985)...... 115 7.2.3 Scene III (1947–1953)...... 116 7.3 Simulating Scenes...... 116 7.3.1 Scene I (1960–1964)...... 116 7.3.2 Scene II (1970–1985)...... 118 7.3.3 Scene III (1947–1953)...... 119 7.4 Spadina Expressway Model (1947–1985)...... 119 7.4.1 Merge Algorithm...... 119 7.4.2 MFULL: The Complete Model...... 122 7.4.3 Analyzing the Complete Model...... 122 7.5 Results and Discussion...... 122 7.5.1 Modeling & Analysis...... 124 7.5.2 Discussion of Results...... 125 7.5.3 Threats to Validity...... 125 7.5.4 Answering the Overall Research Question...... 126 7.6 Summary and Related Work...... 126

IV Conclusions & Future Work 128

8 Conclusions & Future Work 129 8.1 Summary of Contributions...... 129 8.2 Limitations and Improvements...... 131 8.3 Future Work: Extended Support for Understanding Change...... 133 8.4 Future Work: Utility of Goal Modeling...... 134 8.4.1 Activities & Assumptions...... 135 8.4.2 Study Design...... 137 8.4.3 Summary...... 140

Bibliography 141

vii Appendices 152

A Waste Managements Example - Analysis Input 153

B Controlled Experiment: Eﬀectiveness and Usability 156 B.1 Protocol...... 156 B.2 Recruitment Materials...... 163 B.3 Video Scripts...... 165

C CSP Encoding 170 C.1 CSP Background...... 170 C.2 Time Point and Goal Evaluations...... 171 C.3 Relationships & Evolving Relationships...... 173 C.4 Evolving Functions...... 179 C.5 NotBoth Evolving Function...... 183 C.6 Constraints between Time Points...... 184 C.7 Conﬂict Avoidance...... 184 C.8 User Evaluations...... 185 C.9 Encoding and Solving an Evolving Goal Graph...... 186

Index 188

viii List of Tables

2.1 Legend of iStar evaluation labels...... 16 2.2 Destination node label resulting from forward propagation of contribution link...... 17 2.3 Automatic resolution of contribution evaluation label...... 17 2.4 Sample backward analysis for contribution links in Figure 2.4...... 18 2.5 Legend of evidence pairs...... 21 2.6 Forward propagation of satisﬁed and denied evidence pairs...... 23 2.7 Algorithm 2.5 intention assignments for the Bike Lanes example...... 32

3.1 Time points associated with the Bike Lanes example...... 37 3.2 List of common compound functions...... 40

4.1 Constraints associated with the Bike Lane example...... 51 4.2 Graph evaluation path result for the Bike Lanes example...... 58

5.1 List of features implemented in GrowingLeaf and BloomingLeaf...... 86

6.1 GrowingLeaf tool versions created for the experiment...... 95 6.2 Videos created for the experiment...... 96 6.3 Controlled experiment protocol...... 97 6.4 Subject data for RQ 6.1...... 100 6.5 Subject data for RQ 6.2...... 101 6.6 GrowingLeaf improvements...... 105

7.1 List of analysis questions for the Spadina Expressway...... 118

ix List of Figures

1.1 Early-RE iStar model for the Waste Management example (WME)...... 5 1.2 Early-RE Tropos model for the Bike Lanes example (BLE)...... 6

2.1 Fragments of the WME...... 13 2.2 Legend of visual goal model syntax used in this dissertation...... 14 2.3 WME-Slice: Model slice of the Waste Management example (WME) from Figure 1.1... 15 2.4 Contribution relationships in WME-Slice...... 18 2.5 BLE-Slice: model slice of the Bike Lanes example (BLE) from Figure 1.2...... 19 2.6 Speciﬁcation of the Bike Lanes example goal graph fragment...... 20 2.7 A lattice of elements in E...... 22 2.8 Propagation rules applied to evidence predicates...... 22 2.9 BLE-Fragment: prior to evaluation...... 30 2.10 BLE-Fragment: after evaluation...... 30

3.1 Mapping of ticks to time points...... 36

3.2 Sequence of atomic functions over time points t0...tend...... 38 3.3 Sequence of atomic functions in Figure 3.2 repeated three times...... 39 3.4 Slice of Bike Lanes example with evolution...... 43 3.5 Speciﬁcation of a fragment of Bike Lane example goal graph with evolving elements.... 43 3.6 Model for the Waste Management example (WME) with evolving function labels...... 45 3.7 An illustration of the User-Defined function type for GW Education Program...... 46 3.8 Speciﬁcation of the Waste Management example (WME) with evolving elements...... 48

4.1 Evaluation of the Bike Lane example at tdecision = 5...... 53

4.2 Evaluation of the Bike Lane example after ta1 = 14...... 53 4.3 Illustration of Algorithm 4.4 with the Waste Management example...... 61 4.5 Speciﬁcation of the full Bike Lanes example goal graph with evolving elements...... 68

5.1 Screenshot of GrowingLeaf with an iStar model of the Waste Management example.... 74 5.2 Screenshot of GrowingLeaf analysis view...... 75 5.3 Constraint view of GrowingLeaf...... 76 5.4 Analysis view showing query function...... 76 5.5 Architecture of GrowingLeaf...... 77 5.6 Model of Bike Lanes example in BloomingLeaf...... 79 5.7 Modeling view with the goal Justify Cost selected...... 79

x 5.8 BloomingLeaf attribute panel...... 80 5.9 Analysis view showing the analysis results at t = 36...... 81 5.10 Absolute and relative assignments view...... 82 5.11 Cycle detection feature highlighting cycle...... 83 5.12 Cycle detection error message...... 83 5.13 Syntax error message...... 84 5.14 Architecture of BloomingLeaf...... 84

6.1 iStar model of GrowingLeaf example with selected analysis labels...... 89 6.2 Scalability tests for ASR1-ASR3...... 92 6.3 Waste Management model, used in RQ2...... 94 6.4 Trusted Computing model, used in RQ 6.1...... 95 6.5 Network Administrator model, used in RQ 6.1...... 96 6.6 The subjects’ level of familiarity with RE and the iStar...... 97 6.7 Stacked bar chart of the subjects’ scores in RQ 6.0...... 98 6.8 The subjects’ evaluation of GrowingLeaf...... 104 6.9 Likert graph of the subjects’ likelihood to recommend GrowingLeaf...... 106

7.1 Spadina Expressway case description...... 111 7.2 Spadina Expressway route map...... 111 7.3 Mapping between wall-clock and simulation time, for each model...... 113 7.4 MPRO : Scene I fragment at t = 4 (Jan. 1962)...... 114 7.5 MOPP : Scene II fragment at t = 12 (Jan. 1976)...... 115 7.6 MPLAN : Scene III fragment at t = 12 (July 1953)...... 117 7.7 Illustration of merge of MPRO and MOPP ...... 120 7.8 Resolving MPRO and MOPP merge of evolving function for Build Lawrence to Eglinton... 121 7.9 MFULL: Spadina Expressway project model (1947–1985)...... 123

xi List of Algorithms

2.1 Update Label ...... 24 2.2 Label Graph ...... 25 2.3 Propagate ...... 27 2.4 Check Prop ...... 28 2.5 Compute Goal Graph Evaluation...... 29

4.1 Check EI Func ...... 55 4.2 Check NotBoth ...... 56 4.3 Compute Evolving Graph Evaluation Path...... 57 4.4 Forward Graph Evaluation Path...... 62

C.1 Encode as CSP & Compute Time Point Path and Goal Graph Evaluation...... 187

xii List of Theorems and Deﬁnitions

2.1 Definition (Goal Model in iStar)...... 13 2.2 Definition (Evaluation Label in iStar)...... 15 2.3 Definition (Intention Evaluation in iStar)...... 15 2.4 Definition (Forward Evaluation in iStar)...... 16 2.5 Definition (Backward Evaluation in iStar)...... 18 2.6 Definition (Goal Graph)...... 19 2.7 Definition (Evidence Pair)...... 20 2.8 Definition (Intention Evaluation)...... 20 2.9 Definition (Conflict Levels)...... 25 2.10 Definition (User Evaluation)...... 26 2.11 Definition (Goal Graph Evaluation)...... 26

3.1 Definition (Intention Evaluation at a Time Point)...... 36 3.2 Definition (Atomic Functions)...... 36 3.3 Definition (Disjoint Neighbouring Intervals)...... 37 3.4 Definition (Compound Function)...... 38 3.5 Definition (Repeating Compound Function)...... 38 3.6 Definition (User-Defined Function)...... 39 3.7 Definition (N-ary Evolving Relationship)...... 40 3.8 Definition (Binary Evolving Relationship)...... 40 3.9 Definition (NotBoth Evolving Function)...... 41 3.10 Definition (Graph Constraints)...... 42 3.11 Definition (Evolving Goal Graph)...... 42

4.1 Definition (Goal Graph Time Points)...... 50 4.2 Definition (User Evaluation at a Time Point)...... 51 4.3 Definition (Goal Graph Evaluation at a Time Point)...... 52 4.4 Definition (Time Point Path)...... 53 4.5 Definition (Graph Evaluation Path)...... 53 4.1 Theorem (Termination)...... 58 4.2 Theorem (Completeness given a Complete Path)...... 59 4.3 Theorem (Soundness)...... 59

C.1 Deﬁnition (Constraint Satisfaction Problem (CSP))...... 170

xiii Part I

Motivation & Prior Work

1 Chapter 1

Introduction

1.1 Preface

Software is closely intertwined with change because everything changes, and dealing with change is a problem in software engineering [137]. Planning for and dealing with changes is important in every part of the software lifecycle [107], with approaches focusing on requirements and the problem space prior to development [80][139]. Understanding change also relates to software estimation [82] and project management [18]. In this dissertation, we consider modeling and reasoning about anticipated changes (i.e., known unknowns), and do not consider unanticipated change. To explain the broader context of this dissertation, we give an overview of anticipated change in three contexts: ﬁrst, behavioural systems that represent the dynamic world; second, modeling anticipated changes and reconﬁguration of system behaviour; and third, modeling anticipated structural changes.

1.1.1 Behavioural Systems

We begin by looking at change (i.e., events) in the software itself. Dynamically changing systems (e.g., software) have observable behaviour that changes over time. We can specify such systems with automata [8, 67] (e.g., state-machines [125]) to determine which inputs are accepted and predict the resulting behaviour. Some of these systems are described with explicit notions of time (e.g., real-time abstract state machines, synchronous automata), while others have an implicit notion of time [57]. In addition to modeling changes in a software system, we also want to verify the correctness of such changes and prove properties (e.g., safety) over the changes. For example, Labelled Transition Systems (LTS) [84], can model sequential programs, while Calculus of Communicating Systems (CCS) [108] can be used for concurrent systems. System requirements and speciﬁcation focuses on capturing these changes in the system [140], and model checking can be used to verify correctness of system behaviour and help debug speciﬁcations [27]. System dynamics also models and makes predictions about system behaviour [43,138]; but it is used to model continuous variables where changes in behaviour are nonlinear (e.g., rates and levels) [138].

2 Chapter 1. Introduction 3

1.1.2 Designing for Anticipated Behavioural Changes

With dynamic systems, we look at observing and describing the behaviour of a change. We now look at adapting entire systems in the event of known changes. When a change occurs, the system can be reconfigured dynamically or as part of a static update. For example, adaptive systems react to changes in their environment by reconfiguring the system’s behaviour [113][131]. Two approaches for this are choosing a variant dynamically or automatic self-reconfiguration (i.e., tuning). When choosing a variant dynamically, the system has a set of predefined variants to choose from. When change can be anticipated at design time, the options for these changes can be built into the software through configuration [16]. Software product line engineering builds the concept of variants into the design of systems [30][87]. A product line is a set of multiple versions. Depending on the desired functionality, a particular variant is chosen. There is no explicit notion of time in software product lines, but in adaptive systems, individual products can be chosen over time. As with dynamic systems, it is important to prove properties over variants in a software product line [29].

1.1.3 Designing for Anticipated Structural Changes

In a software system, not all changes are behavioural. It is well documented that software maintenance consumes significant resources in the software lifecycle [95][50]. We also consider changes to the architecture or structure of the system as well as conceptual changes and broader changes in the business of software. We can describe structural changes with a temporal dimension (i.e., describing changes that will happen in the future). Newly found uses for systems cause them to evolve and their architecture to drift [54]. For example, we can describe and make predictions about the context of a system in the future. As introduced above, software product line engineering looks at variants in the behaviour of software, but it also can be used to consider variants in the software architecture and system level features that dictate the structure and configuration of software systems [55][118]. It is valuable to model changes in software organizations as well because they are dynamic and exist in a constantly changing world [65]. In software development, technical and social factors result in changes [100]. We also consider changes in the beliefs of stakeholders [36], as well as changes in dependency relationships between stakeholders. Software exists within a broader context of the business organization. High-level business objectives may change over time affecting both the goals of the software as well as the process of building it [20][22][81][97][115]. The work described here explores enabling stakeholders to make predictions about structural changes in a project with and without a temporal component. However, this work does not address how to describe changes in the intentions of stakeholders and changes of stakeholders themselves. We are attempting to describe and simulate conceptual decisions based on anticipated changes in the context of early-phase requirements models. In the next section, we introduce decision making in early phase requirements engineering.

1.2 Decision Making in Early Requirements

In early-phase requirements engineering (RE), goal modeling helps stakeholders understand and evaluate potential project scenarios, and the system’s interactions with its surrounding environment. Speciﬁcally, goal modeling elicits and connects stakeholders’ intentions and social needs with technical requirements. Chapter 1. Introduction 4

Goal modeling has long been used in the literature to model and reason about system requirements, constraints within the domain and environment, and stakeholders’ goals [5, 53, 140, 147]. Within the goal-oriented requirements engineering (GORE) community, different approaches and notations have been developed, e.g., iStar (originally called i*) [147], NFR [26], GRL [5], Tropos [53], and KAOS [140]. These approaches differ in their purpose and application. For example, NFR and iStar deal with only qualitative information, while the rest can also evaluate quantitative information. Tropos and KAOS do not fully support dependencies between actors but have formal semantics. GRL and Tropos have automated analysis procedures. See Horkoff and Yu’s work for a more comprehensive comparison of these approaches [75]. The principal artifact of these approaches is the goal model, which is a visual representation of functional and non-functional requirements of the project with or without an explicit notion of a stakeholder. While some approaches have a textual representation (e.g., iStarML [21], GBRAM [9]), the process of goal modeling is dominated by the creation and analysis of the visual representation. In this dissertation, we focus on understanding the non-functional requirements (NFRs) of stakeholders in early-RE, with the aim of helping stakeholders collaborate by understanding each other’s intentions and dependencies. Considering the many GORE approaches, we first chose to extend iStar because it represented both the concepts of actors and non-functional requirements well, and then chose to extend Tropos because of its formal foundations for analysis and similarity in syntax and propagation to iStar. For the purpose of discussing the details of the framework, not all models in this dissertation present actors and their boundaries; but, the foundational target of this research is still to enable discussions between and about stakeholders. In choosing these languages we were able to take advantage of their propagation-based analysis. Many techniques for analyzing goal models already exist [76]. Generally, propagation-based approaches either determine the evaluation of root-level intentions given evaluation assignments to leaf- level intentions, called forward analysis (or forward propagation), or determine possible evaluations of leaf-level intentions given evaluation assignments of root-level intentions, called backward analysis (or backward propagation). In making trade-off decisions, forward analysis is used to ask “what if” questions by determining the impact of selecting a set of alternatives, whereas backward analysis is used to generate possible scenarios that result in the fulfillment of one or more high-level goals. Variations of forward and backward analysis have been created for interactive qualitative analysis [76], and automated qualitative and quantitative analysis [6, 53]. However, these techniques are insufficient for answering time-based questions, and may give incorrect answers if the fulfillment of intentions changes in the future, as we illustrate in the following two subsections.

1.2.1 Motivating Example I: Waste Management Example (WME) in iStar

Consider a city evaluating its waste management infrastructure. The city currently has a landﬁll (dump) that has not reached capacity and is considering investing in building a new one as well as a recycling and composting facility. Figure 1.1 shows the partial Strategic Rationale iStar goal model [147] of the City and its Citizens, consisting of their intentional elements (intentions): goals, tasks, resources, and soft- goals; and connections between their intentions: dependency, decomposition, and contribution links (see Legend in Figure 1.1 for representations of intentions and connections, and Chapter2 for an overview of iStar). The city wants to satisfy Manage City Waste, Comply with Standards, Reduce Operating Costs, Have Workers Union Contract, and for their citizens to Enjoy City (i.e., their root-level goals). Chapter 1. Introduction 5

Figure 1.1: Early-RE iStar model for the Waste Management example (WME).

Several elements in this model change over time. For example, the Willingness to Separate Waste will become more satisfied over time, but Environmental Concern will vary substantially depending on external factors. As the dump gets used, the Space in Dump will become less satisfied, and if the city decides to build a new dump at some point, it will need to Purchase Land either near the city where prices are high or far away from the city, where prices are lower, or both. Buying land farther away from the city results in increased garbage transportation costs. Modeler Questions. The city wants to understand how these goals changing over time impact its decision making, by exploring the following six questions: WME-Q1 Is it feasible to first Build Green Centre and then build a dump (Build Large Dump or Build Small Dump)? WME-Q2 Does the order of these developments (Process Green Waste and Use New Dump) matter? WME-Q3 Which possible scenarios always satisfy Manage City Waste even if Space in Dump becomes denied in the future? WME-Q4 How do changes in Environmental Concern affect the city’s root-level goals over time? WME-Q5 Can all root-level intentions eventually be satisfied? WME-Q6 Given the initial results from answering WME-Q1-WME-Q5, is there another way to answer these questions? Current Approaches. Current goal model analysis approaches can answer some of the City’s questions to a limited extent. For example, we could use forward analysis to answer WME-Q1 (and, similarly, WME-Q2) by creating an initial model before anything is built and additional models for each combination of Build Green Centre, Build Small Dump, and Build Large Dump, but manual comparison of these models is fraught with errors and does not account for the elapsed time between models. Forward analysis can also be used for WME-Q4 by exhaustively propagating evaluation labels, but again it does not say how Chapter 1. Introduction 6

Figure 1.2: Early-RE Tropos model for the Bike Lanes example (BLE).

the individual labels connect, and comparison between propagated models is difficult. Backward analysis can partially answer WME-Q3, by providing possible solutions with and without the denial of Space in Dump, and WME-Q5 can also be answered by setting all root-level goals to be satisfied and finding values for the remaining intentions using backward analysis. However, backward analysis is not able to inform stakeholders how (or in what order) to satisfy these goals. In each of these and other cases, standard techniques cannot explicitly answer the city’s questions and cannot provide a path to achieve their satisfaction values. Furthermore, no current approaches explicitly allow the exclusion of multiple previous solutions (WME-Q6), which is vital in grasping the full range of tradeoffs. Time-based approaches can answer these questions by automatically generating future models and propagating values over time intervals, removing the need for manual comparison.

1.2.2 Motivating Example II: Bike Lanes Example (BLE) in Tropos

Urban roads are often overcrowded leading to collisions between motorists and cyclists. Proponents of cycling argue for the addition of separated cycling lanes (called bike lanes). We consider a project to add bike lanes to a major residential and commercial thoroughfare in a large city. City planners are considering whether to install the lanes and if so, how different options, specifically, whether they should be installed curb-side or have parking curb-side. Another tradeoff they considered is whether to invest in permanent lanes (i.e., involving road work) or temporary lanes (i.e., re-painting lines). The city is considering the impacts of these changes on pedestrians, cyclists, motor vehicles, transit, road operations, commercial demands, and emergency services, through the goals in the model. Figure 1.2 shows a partial goal model of the project tradeoffs in Tropos (see Chapter2 for an overview Chapter 1. Introduction 7 of Tropos). The city planners want to satisfy Have Bike Lanes while satisfying Cyclist Safety and Access to Parking (i.e., some of their root-level goals). Several elements in this model change over time. For example, Have Design becomes more satisfied at some point, but Bike Lane Usage may vary depending on external factors, such as seasonal changes in weather. Permanent Construction affects Access to Parking while the roadway is being reconfigured, whereas Temporary Construction does not have the same effect. An assumption in this model is that the city planners have already decided that the No Parking design option is not feasible because of its negative effect on Access to Parking and Public Support. We include this option for completeness but do not consider it in further tradeoff analysis. Modeler Questions. The city planners want to understand how changes in these goals over time impact their decisions by exploring the following questions: BLE-Q1 Are bike lanes or parking most effective on curb-side, given seasonal constraints on road operations? BLE-Q2 Is a permanent solution or a temporary solution most appropriate, given possible changes to cyclist and motor vehicle traffic? BLE-Q3 How do variations in the satisfaction of Bike Lane Usage over time affect the City’s goals. BLE-Q4 What scenario ensures that Access to Parking is satisfied in 36 months, even if off-street lots become unavailable? BLE-Q5 Can the City eventually satisfy and maintain Cyclist Safety? BLE-Q6 Can the City delay the design and planning decisions to the future? Current Approaches. Standard forward and backward analysis can be used to explore BLE-Q1 and BLE-Q2, but may not result in the best decision if changing intentions are not considered. For example, we can use forward analysis to answer BLE-Q1 (and similarly BLE-Q2) by creating an initial model before anything is built and additional models for each combination of Bike Lane Curbside and Parking Curbside, but manual comparison of these models is error prone and does not include elapsed time between models. Forward analysis can also be used for BLE-Q3 by exhaustively propagating evaluation labels, but again this does not say how the individual labels connect, and comparison between propagated models is difficult [61]. Backward analysis can partially answer BLE-Q4 by providing possible solutions with and without the satisfaction of Off-street Lots, and BLE-Q5 (similarly for BLE-Q6) can also be answered by setting Cyclist Safety to be satisfied and finding values for the remaining intentions using backward analysis. However, backward analysis is not able to inform stakeholders how (or in what order) to satisfy these goals.

1.2.3 Gaps Identiﬁed In This Dissertation

We begin by reviewing related work as of 2015, the initial publication of the ideas in the dissertation [58].

Changing Goals. Regev and Wegmann discuss how goals change in their achievement over time for the purpose of developing heuristics to improve goal identification [119]. Using KAOS, Letier et al. consider goals that are real-time properties of a system and develop a formal approach to construct software specifications from these goals [92]. They define the goals in temporal logic and connect timed event- based models with state-based formalisms [91]. Goncalves et al. look at conditional non-monotonic temporal beliefs and goals. They represent and reason with them using an ASP solver and an LTL reasoner [56]. Liaskos et al. used a linguistics framework to discover variability in goal models, Chapter 1. Introduction 8 specifically temporal requirements about the duration and frequency of events [94]. In our work, we look at the evolution of goals rather than looking at concrete timing requirements, or concrete events within the model.

Simulation of Agents. Gans et al. discuss the simulation and operationalization of agents’ goals and multi-agent systems, in agent-based approaches [48]. In our work, we are considering scenarios about top-level goals and actor intentionality, rather than the tasks completed by collections of agents. Cheong and Winikoﬀ focus on connecting time-dependent goals within a decomposition (denoted by precondition links), allowing for the coordination and achievement of agents’ plans [24, 25]. Gans et al. combine iStar with ConGolog to create a utility-based temporal ordering for agents to execute tasks in order to simulate goal alternatives [47]. Instead, we use ordered dependency constraints speciﬁed by the modeler.

Requirements Evolution. Webber and Gomaa look at developing requirements for evolution, but with the goal of developing assets for reuse between systems by modeling variation points [142]. Anton and Potts consider the evolution of goals for redesigning legacy systems [9]. Ernst et al. look at incrementally updating previous solutions as requirements evolve [41]. Hartmann et al. support modeling evolution of each element independently by capturing element change histories [66]. Instead, we analyze evolution of stand-alone systems prior to them being built.

Goals outside early RE. Goals as dynamic entities have been used outside the requirements phase for runtime analysis and monitoring [15, 46, 121], self-adaptive systems [14, 133], testing [37], and context- aware systems [141]. Speciﬁcally, Dalpiaz et al. distinguish between design-time goal models and runtime goal models. Our models remain design-time entities throughout our analysis [34]. Robinson et al. convert goal models into components that monitor states in the system through runtime monitoring [121]. Baresi et al. add adaptive goals to goal models [14]. These approaches do not allow stakeholders to explore how the requirements in the model change, but rather check properties of the system as it runs.

Gaps. Approaches to early-phase RE do not explicitly model how the evaluation (i.e., level of satisfaction or fulfillment) of a goal changes over time. When making trade-off decisions, stakeholders can, in principle, reason about time in an ad-hoc manner, by considering the state of the model before or after the change. But this is insufficient to understand trends over time. That is, goal models without an explicit modeling of time may lead to incorrect results because they do not account for variations in intentions’ satisfaction. It is important to provide support to allow stakeholders to consider the impact of alternatives over time. In this dissertation, we allow the evaluation of intentions to change over time and enable stakeholders to answer time-based questions by creating simulation paths.

More Recent Work. Prior to discussing our contributions, we highlight recent advances published after our initial position paper [58]. Aprajita and Mussbacher provide a quantitative framework (called TimedGRL) that enables stakeholders to consider possible evolutions and trends through visualizations [12]. In the associated thesis, Aprajita provides a detailed example of how TimedGRL can be applied to review the performance of a project [10]. In subsequent work, called TimedURN, Aprajita et al. add support for feature models (i.e., Use Case Maps (UCM)) [11] and provide extensive tooling [99]. To the best of our knowledge, our approach currently remains the only one that enables modelers to generate and explore future evolutions of stakeholders’ intentions. Chapter 1. Introduction 9

1.3 Contributions

In this dissertation, we consider the problem of evaluating project trade-oﬀs when the satisfaction of stakeholders’ intentions (i.e., goals, tasks, resources, and soft-goals) changes over time. We hypothesize that goal modeling and by extension decision making in early-phase RE can be improved by adding temporal components to accommodate changes in intentions, enabling stakeholders to reason about model evolution. Our central research question is

RQ 1 How can we eﬀectively model, reason with and support users in making tradeoﬀ decisions given evolutionary information.

In this dissertation, we address this research question by creating and validating a framework (entitled the Evolving Intentions framework) for modeling and reasoning about intention evaluations that change over time for the purpose of answering evolutionary questions in early-phase RE. The core concept of Evolving Intentions is to vary evaluation labels of intentions within a goal model over time to make trade-off decisions given future evolutions. We examine the problem, present the framework, and report on tools built to evaluate the proposed solution. As mentioned in Section 1.2, iStar and Tropos were developed for different purposes and have different semantics. We introduce the Evolving Intentions framework in the context of both the iStar and Tropos modeling languages to explore the applicability to both the informal notation of iStar (which allows non-technical users to easily explore goals), as well as the more formal methodology of Tropos (which connects with downstream development activities). We demonstrate the framework using the Waste Management Example for iStar, and the Bike Lanes Example for Tropos. Both examples could have been modeled in either language. Since we use the examples purely to illustrate the respective formalisms, we did not repeat each example in both languages. We explicitly chose to present different examples because we believe separate examples provide a cleaner presentation for consistency across chapters of this dissertation.

1.3.1 Modeling Evolving Intentions

In Chapter3, we consider how to represent changes in evidence for the satisfaction of intentions in goal models in the context of both goal modeling languages (i.e., Tropos and iStar) using qualitative evaluation labels. Thinking about real world examples, we observe common patterns of satisfaction, and generate a set of functions for Evolving Intentions both for a single time period and for multiple time periods. Our functions can be used as building blocks to generate compound functions with repeating behaviour. We also extend the notion of evolution to relationships in goal models, where relationships can exist only before or after a time point. Additionally, we allow the type of contribution and decomposition links to change over time. In doing so, we contribute terminology that enables stakeholders to discuss how intentions may evolve. Functions for Evolving Intentions can be assigned to goals in a static goal model.

1.3.2 Reasoning with Evolving Intentions

In Chapter4, we contribute reasoning to answer questions about future evolutions of goal models and time-based trade-offs. In this dissertation, we require that models do not contain cycles in their links, and we treat each goal model as a collection of directed acyclic graphs. Our initial approach, Simulation Chapter 1. Introduction 10 over Evolving Leaf Intentions, builds directly upon forward analysis, meaning that at each time point the evaluation labels of leaf-level nodes are propagated to root-level nodes. In this case, a generated simulation is a set of ordered time points (relative time), and allows us to determine the future evolution of a given scenario. To enable stakeholders to generate possible simulations that lead to the satisfaction of a specific goal, we extend backwards analysis allowing stakeholders to assign evolving functions to root-nodes and generate a path that resulted in the satisfaction of these root-nodes. By combining forward and backward propagation in Simulation over All Evolving Intentions, evolving functions can be assigned to any node in the model. To connect these simulations with the intuitions of stakeholders, we propose three simulation strategies, and consider analysis over relative and absolute time. This allows stakeholders to create a random path, given evaluation labels at the current time point, evolution information for the model, and desired properties of the initial, intermediate, and final states of the model. We also extend this analysis to allow stakeholders to create a path which is different than the previously seen one over the same constraints. In Section 4.1, we define the notion of an evolving goal graph path and prove properties of the analysis. This allows for traceability of results with a unique interpretation of values, resulting in better answers to time-based trade-offs.

1.3.3 Tool Support for Modeling and Reasoning

In Chapter5, we describe two web-based tools that implement the modeling and analysis contributions discussed above. The value of the Evolving Intentions framework is demonstrated in GrowingLeaf for iStar, and BloomingLeaf for Tropos. Both tools are built on top of the same canvas and diagraming libraries; therefore, they have a similar architecture, as well as the same look and feel. Since the reasoning capabilities are diﬀerent, we describe the unique features and usability of each tool. For example, both tools compute simulations by combining forward and backward analysis. Since backward analysis is non-deterministic, we implemented our path-based exploration as a constraint satisfaction problem (CSP) [49][128]. GrowingLeaf also includes the ability to generate simulations based on forward analysis. The primary contribution of these tools is that they enable us to validate diﬀerent aspects of our framework including automated analysis.

1.3.4 Validating Modeling, Reasoning, and Tooling

In Chapter6, we validate the Evolving Intentions framework with iStar. In Section 6.1, we report on an initial validation of our representation and reasoning for Evolving Intentions by building and analyzing a large model of the intentions of the GrowingLeaf development team. We also assess the scalability of our approach by reporting runtimes of our analysis on randomly generated examples and models created for this dissertation. In Section 6.2, we investigate the effectiveness and usability of Evolving Intentions, Simulation over Evolving Leaf Intentions, and GrowingLeaf, and report on a between-subjects experiment we conducted with fifteen graduate students familiar with requirements engineering. We found that our representation of Evolving Intentions is suitable to the task of identifying and representing intentions over time, and that Simulation over Evolving Leaf Intentions improves stakeholders’ ability to reason about goal models over time, but only when stakeholders’ effectively selected evolving functions. We conclude that GrowingLeaf is found to be usable and effective by the subjects. Chapter 1. Introduction 11

In Chapter7, we validate the Evolving Intentions framework with Tropos. We use historical data and rational reconstruction to understand how the Toronto Spadina Expressway project evolved in the past. We model the project over diﬀerent time periods, which enables us to assess the modeling and simulation capabilities of the Evolving Intentions approach, its support for asking and answering ‘what if’ questions, and the maturity of BloomingLeaf. We demonstrate that the Evolving Intentions framework was successful in capturing all of the changes required to understand the Spadina Expressway case. We also demonstrate that the Evolving Intentions framework with path-based analysis was able to fully reproduce the timeline of actual events, as documented based on our rational reconstruction, and alternative futures.

1.4 Thesis Organization

The remainder of this dissertation is organized as follows: • Chapter2 lays down the foundation for the dissertation. We introduce the visual syntax for goal models and give an overview of previous work on reasoning with goal models.

• Chapter3 presents the modeling contributions discussed in Section 1.3.1. This chapter includes content published in [58, 60].

• Chapter4 presents the reasoning contributions discussed in Section 1.3.2. This chapter also includes content published in [58, 60].

• Chapter5 reports on the implementation of two tools built to explore Evolving Intentions, introduced in Section 1.3.3. This chapter extends and combines the work reported in [62, 64].

• Chapter6 gives evidence for the usability of Simulation over Evolving Leaf Intentions and the intuitiveness of the deﬁned functions to describe Evolving Intentions in iStar. Using the work in [60], we demonstrate the use and scalability of Evolving Intentions. Building on the work in [61], this chapter describes our initial validation, via a controlled experiment, of the framework and associated tooling (i.e., GrowingLeaf).

• Chapter7 gives evidence for the eﬀectiveness of the modeling and simulation capabilities of Evolv- ing Intentions in Tropos. We present a rational reconstruction to understand how a project evolved in the past. This chapter is currently under review for publication [63].

• Chapter8 provides a summary of the contributions, and a discussion of limitations and future work (including content from [59]). Chapter 2

Preliminaries

In this chapter, we present relevant goal modeling background required for describing the contributions in the remainder of this dissertation. The core of requirements engineering (RE) [140] is the elicitation of functional requirements for a system and constraints within the environment. We focus on studying problems in early-phase RE where stakeholders are exploring the problem domain as well as possible options for solutions. Goal-oriented requirements engineering (GORE) [122] aims to understand stakeholders and their social relationships within projects that may impact designs and alternatives of the project. While there are many GORE approaches (e.g., NFR [26], iStar [147], and KAOS [140]), in this dissertation, we focus on the iStar language, and languages that grew out of iStar (e.g., Tropos [53] and GRL [5]). Speciﬁcally, we contrast iStar with Tropos to explore the impact of variations in modeling syntax and semantics. We focus on the graphical representation and usability aspects of goal modeling when discussing iStar, and focus on the syntax and semantics of the language when discussing Tropos.

Organization

The remainder of this chapter is organized as follows: Section 2.1 introduces the syntax and semantics of iStar, following the presentation in [68]. Section 2.2 introduces the syntax and semantics of Tropos, following the presentation in [134]. Section 2.3 provides a summary of the chapter.

2.1 iStar Syntax and Semantics iStar is the Intentional STrategic Actor Relationships language (originally named i*) [147]. iStar (and GORE approaches more generally) express social relationships between actors within a project domain, with and without a proposed system. This dissertation directly extends Horkoff’s work on interactive analysis of iStar goal models [68]. In this section, we adapt previous definitions of components of the iStar language [74,77]. Note that iStar has ‘very light’ semantics; thus, we give only a minimal presentation in this section and provide a more thorough formalization of Tropos in the next section. To give a concrete meaning to the concepts, we use a fragment of the model from the Waste Management example (WME), first introduced in Section 1.2.1, to explain the details of iStar. The relevant WME model fragment is shown in Figure 2.3.

12 Chapter 2. Preliminaries 13

Figure 2.1: Fragments of a iStar Strategic Dependency model of the Waste Managements Example.

2.1.1 iStar Overview By Example

Consider the Waste Management example (WME) introduced in Section 1.2.1, where the city is evaluating waste management options, and considering building a dump and recycling facility. In this example, we consider two actors: the City and its Citizens. Strategic Dependency diagrams model dependencies between actors completing work activities. A fragment of a Strategic Dependency diagram for WME is shown in Figure 2.1, where the City and its Citizens both depend on each other. The City and its Citizens want to complete the Separate Recycling task and to do a good job, modeled as the Quality Waste Separation soft-goal. The Citizens depend on the City to provide training for them to understand how the recycling should be separated, which is modeled as the task Complete Waste Separation Training. In this diagram actors remain closed circles. To further understand what motivates these actors we model their rationale. Strategic Rationale diagrams model the intentions (i.e., goals, tasks, resources, and soft-goals) that rationalize the dependency relationships between actors. Actors are not represented with an actor boundary showing the intentions within each actor. The City’s intentions are made explicit in the Strate- gic Rationale diagrams for WME shown in Figure 1.1. As described in Section 1.2.1, we see the City’s goal to Manage City Waste and Reduce Operating Costs, while considering whether to build each combination of Build Green Centre, Build Small Dump, and Build Large Dump. If the City completes Build Green Centre, they still depend on the Citizens for separating waste. While dependency relationships can be expressed in both Strategic Dependency and Strategic Rationale diagrams, Strategic Dependency diagrams are most appropriate for understanding and exploring social relationships, and Strategic Rationale diagrams are most useful for making trade-oﬀ decisions. A hybrid diagram can also be used where some actors are open (i.e., having an actor boundary with intentions) and some actors are closed. In this dissertation, we investigate Strategic Rationale diagrams (and hybrid diagrams) and do not explicitly use Strategic Dependency diagrams, concentrating on how the evaluation of intentions can change over time.

2.1.2 Model Syntax

Strategic Rationale diagrams (here in known as Goal Models), consist of actors and their intentions as well as links between actors/intentions. Chapter 2. Preliminaries 14

(a) Set of intentions. (b) iStar links. (c) Tropos links.

Figure 2.2: Legend of visual goal model syntax used in this dissertation.

Deﬁnition 2.1 (Goal Model in iStar). An iStar model is a tuple hA, G, Ri, where A is a set of actors, G is a set of intentions, and R is a set of relationships between intentions [74, 77]. R is a set of tuples r h(g1, ..., gn), r, gi, often written as (g1, ..., gn) −−−→ g, where r is a relationship type. g1, ..., gn are called source intentions, and g is called a target intention. For n-ary intention relationships, r ∈ {and, or}. Binary relationships have only one source intention and r ∈ {makes, helps, hurts, breaks, depends}.

Intentions are specified as a tuple huniqueName, typei, where uniqueName is the intention’s name that acts as a unique identifier, type ∈ {goal, task, soft-goal, resource}. Actors are specified as a tuple huniqueName, type, Gai, where uniqueName is the actor’s name that acts as a unique identifier, type ∈

{actor, agent, role, position} and Ga is a set of intentions that belong to an actor.

Figure 2.2(a) gives the visual syntax for the four intention types (i.e., goals, tasks, resources, and soft-goals), and actors with an actor boundary. Actors can have relationships between them based on their type (e.g., Role or Agent). We do not use actor types or relationships (see [146] for details) in our analysis. Each intention represents diﬀerent concepts. For example, Goals represent wishes of stakeholders or perceived needs of a system to be. These goals can be achieved through completing activities (i.e., tasks), using resources which include information or material objects. Soft-goals deal with non-functional requirements of the system, and represent qualities where there are no clear-cut criteria for satisfaction or achievement. Chapter 2. Preliminaries 15

Figure 2.3: WME-Slice: Model slice of the Waste Management example (WME) from Figure 1.1.

For example, the speciﬁcation of the WME model fragment in Figure 2.3 is hA, G, Ri where, A = {hCity, actor, {Manage City Waste, Use Current Dump, Use New Dump, Build Small Dump, Build Large Dump, Process Green Waste, Reduce Operating Costs}i}, G = {hManage City Waste, goali, hUse Current Dump, taski, hUse New Dump, taski, hBuild Small Dump, taski, hBuild Large Dump, taski, hProcess Green Waste, goali, hReduce Operating Costs, soft-goali}, or R = {(Use Current Dump, Use New Dump) −−−→ Manage City Waste, or (Build Small Dump, Build Large Dump) −−−→ Use New Dump, helps hurts Build Small Dump −−−→ Reduce Operating Costs, Build Large Dump −−−→ Reduce Operating Costs, hurts Process Green Waste −−−→ Reduce Operating Costs}

2.1.3 Intention Evaluation

Definition 2.2 (Evaluation Label in iStar). The qualitative evaluation labels are Fully Satisfied (FS), Partially Satisfied (PS), Partially Denied (PD), Fully Denied (FD), Conflict (CF), Unknown (U), and None (N).

In iStar, intentions (i.e., goals, tasks, resources, and soft-goals) are the elements in the model that can be assigned qualitative evaluation labels. Table 2.1 gives the mapping between the evaluation label and its symbol and visual syntax. In the remainder of the text, we refer to the evaluation labels by their symbols. Figures use visual syntax, where in the absence of other labels, an intention has the value None.

Deﬁnition 2.3 (Intention Evaluation in iStar). An intention evaluation is a pair hg, li, where g is an intention and l is an evaluation label.

Evaluation labels can be assigned by modelers and can be propagated throughout the model via the relations between intentions (i.e., links). Evaluation labels have the following ordering FS > PS > U(C) > PD > FD, from most satisﬁed to least satisﬁed, where > can loosely be inferred as “more true”.

2.1.4 Relationship Semantics and Propagation iStar relationships are deﬁned in Deﬁnition 2.1. iStar models consist of decomposition, dependency, and contribution links. Decomposition links separate intentions into sub-components. A single intention can Chapter 2. Preliminaries 16

Table 2.1: Legend of iStar evaluation labels.

Name Symbol Visual Syntax (Fully) Satisfied FS Partially Satisfied PS Partially Denied PD (Fully) Denied FD Conflict C Unknown U None N

be decomposed with either And-Decomposition or Or-Decomposition links but not both. Figure 2.2(b) gives the visual syntax for the relationships in an iStar model. In propagation, And-Decomposition (and in Figure 2.2(b)) propagates the minimum values of the sub-components, and Or-Decomposition (or in Figure 2.2(b)) propagates the maximum values. We do not distinguish between Or-Decomposition and means-ends links, which are used to specify that a task is a way to achieve the goal. Dependency links (depends in Figure 2.2(b)) show directed dependency relationships between actors. The source actor (depender) depends on another actor (dependee) for an intention (dependum). Contribution links propagate values to soft-goals. There are four types of contribution links: makes, helps, hurts, and breaks (see Figure 2.2(b))1. In this dissertation, evaluation labels are either assigned to each intention in a goal model by the modeler or through propagation. iStar has two types of propagation-based analysis for evaluating goal models: forward analysis and backward analysis. Forward Analysis. Forward analysis gives stakeholders the ability to ask ‘what if’ questions. Using forward analysis, a modeler assigns evaluation labels to leaf-level intentions and then propagates the evaluation labels to root-level intentions [77].

Deﬁnition 2.4 (Forward Evaluation in iStar). Let a goal model hA, G, Ri, and a set of intention evaluation E for a leaf-level intention be given. A complete forward evaluation of hA, G, Ri is the total assignment of all intentions in the model via forward propagation across all relationships in R, such that ∀g ∈ G : g ∈ E.

In forward analysis, propagation depends on the type of relationship. And-Decomposition takes the minimum value of all the child nodes, and Or-Decomposition takes the maximum value of all the child nodes, given the ordering FS > PS > U(C) > PD > FD. For example, in Figure 2.3, if Build Large Dump is PD and Build Small Dump is PS then the Or-Decomposition link will propagate PS to Use New Dump, because PS > PD. Contribution links propagate the values depending on the type of link as speciﬁed in Table 2.22. Once values have been propagated, an intention with more than one incoming contribution link will have multiple values. We resolve the evaluation label by applying the rules in Table 2.3 pair-wise to the

1In this dissertation, we exclude some+, some-, and unknown contribution links [26]. 2We do not propagate conﬂicting values in forward analysis [6]. Chapter 2. Preliminaries 17

Table 2.2: Destination node label resulting from forward propagation of contribution link.

Resulting Target Evaluation Label Based On Contribution Type makes helps hurts breaks FS FS PS PD FD PS PS PS PD PD U U U U U C U U U U PD PD PD PS PS Source Label FD FD PD PS PS N N N N N

Table 2.3: Automatic resolution of contribution evaluation label.

Label B FS PS U C PD FD N FS FS FS U C C C FS PS FS PS U C C C PS U UUUCUUU C CCCCCCC PD C C U C PD PS PD Label A FD C C U C PS FD FD N FS PS U C PD FD N

intentions. For example, we want to determine the evaluation label of Reduce Operating Costs in Figure 2.4. Build Small Dump and Process Green Waste have the value FS, and Build Large Dump has the value FD. Using the forward analysis contribution table (see Table 2.2), Reduce Operating Costs will receive three values: PS from Build Small Dump, PS from Build Large Dump, and PD from Process Green Waste. Using Table 2.3, we resolve the values {PS, PS, PD} as C ; thus, Reduce Operating Costs is given the conflict label C . Dependency links propagate the value from dependee element to the depender element, because the depender depends on the evaluation of the dependee’s intentionality. Thus, in forward analysis, the evaluation is propagated in the opposite direction of the link. Forward analysis was originally intended to be completed iteratively with stakeholder involvement for the resolution of intentions [68]. For example, in our discussion of Figure 2.4, a stakeholder could have resolved the conflict by declaring that Reduce Operating Costs should be assigned a PS label. In this dissertation, we use an automatic resolution of contribution labels. Backward Analysis. Backward analysis gives stakeholders the ability to find scenarios where desired goals can be achieved. Using backward analysis, a modeler assigns an evaluation label to each root intention and then propagates the evaluation labels to leaf intentions. Chapter 2. Preliminaries 18

Figure 2.4: Contribution relationships in WME-Slice.

Table 2.4: Sample backward analysis for contribution links in Figure 2.4. PS is the desired evaluation label for Reduce Operating Costs.

Test # Items 1 2 3 Build Small Dump FS FD FS Build Large Dump FS FD FD Process Green Waste FS FD FD Build Small Dump + helps PS PD PS Build Large Dump + hurts PD PS PS Process Green Waste + hurts PD PS PS Propagated Reduce Operating Costs Label C C PS Valid Reduce Operating Costs Label? no no yes

Deﬁnition 2.5 (Backward Evaluation in iStar). Let a goal model hA, G, Ri, and a set of designed intention evaluation W = {hg0, l0i, ..., hgn, lni} for root-level intentions be given. A complete backward evaluation of hA, G, Ri is the total assignment of all intentions E in the model via forward propagation across all relationships in R, such that ( ∀g ∈ G : g ∈ E) ∧ ( ∀g ∈ W : g ∈ E ∧ W (g) = E(g)).

Backward analysis iteratively selects potential labels for a node and then uses the forward analysis rules to check if the selected potential labels are valid assignments. We illustrate this process with the fragment in Figure 2.4. Suppose we want Reduce Operating Costs to be partially satisfied. In backward analysis, we assign labels to root-level intentions; thus, we assign Reduce Operating Costs the value PS. We then assign potential labels to leaf-level nodes and forward propagate them to determine if they are valid assignments. If we assume each of the leaf-level intentions in Figure 2.4 can have one of four values (i.e., FS, PS, PD, and FD), then there are 43 possible tests. Table 2.4 lists three tests consisting of sets of potential labels. These tests were selected randomly for illustrative purposes. The first two tests did not result in a PS assignment of Reduce Operating Costs; but, in test 3, a valid solution was found where Reduce Operating Costs was assigned a PS label. Our backward analysis is based on the implemented automated analysis algorithm in jUCMNav [7] and the work of Luo and Amyot [98]. In Section 2.2, we discuss a brute force algorithm for selecting intention evaluations. This algorithm can be applied to finding evaluation labels for iStar goal models as well. Chapter 2. Preliminaries 19

Figure 2.5: BLE-Slice: model slice of the Bike Lanes example (BLE) from Figure 1.2.

2.2 Tropos Syntax and Semantics

Tropos (named from the Greek word “trope” meaning “easily changeable”) is a methodology that takes actors and agents as core paradigms and uses them throughout the entire software development process [19]. Tropos has diagrams and concepts to cover early and late-phase RE, architectural design, and detail system design and implementation. The Tropos methodology uses the iStar language for modeling actors and their intentions, having similar concepts. For example, Strategic Dependency diagrams are called Dependency Models and Strategic Rationale diagrams are called Actor Models, with hybrid diagrams called Mixed models. Tropos uses different syntax and semantics for model links as well as evaluation, which functions as a better foundation for automated analysis, but the overall purpose and concepts correspond to those introduced for iStar above. We use the formalism presented by Giorgini and collaborators as the basis for our work [51,52,134]. In this section, we present and adapt these definitions accordingly. For example, to maintain consistency with the work of Giorgini et al., a goal model is referred to as a goal graph. To give new concepts a concrete meaning, we use a fragment of the model from the Bike Lanes example (BLE), first introduced in Section 1.2.2. The BLE model fragment is shown in Figure 2.5. Since the focus of this dissertation is on the analysis of changes in intention evaluations over time, we periodically ignore the distinction between intention types and exclude actor boundaries, as in Figure 1.2 and Figure 2.5.

2.2.1 Model Syntax

We ﬁrst introduce the formal syntax and semantics of the goal model.

Deﬁnition 2.6 (Goal Graph). A goal graph is a tuple hA, G, Ri, where A is a set of actors, G is a set r of intention nodes and R is a set of tuples h(g1, ..., gn), r, gi often written as (g1, ..., gn) −−−→ g, where r is a relationship type and g, g1, ..., gn ∈ G. g1, ..., gn are called source intentions, and g is called a target intention. For n-ary intention relationships, r ∈ {and, or, no}. Binary relationships have only one source intention, and r ∈ {+S, -S, +D, -D, ++S, --S, ++D, --D, +, -, ++, --, no}.A well-formed goal graph is a graph where: (1) each intention has at most one incoming n-ary relation, and (2) the graph does not contain any loops. Chapter 2. Preliminaries 20

As shown in Figure 2.2, decomposition links are the same as in iStar as in Tropos with and and or decomposition. Contribution links are one of twelve types: {+S, -S, +D, -D, ++S, --S, ++D, --D, +, -, ++, --} (see Figure 2.2(c)). ++/+/-/-- links are symmetric and roughly map to iStar contribution links. S links only propagate evidence when the source intention is satisﬁed, while D links only propagate evidence when the source intention is denied. Although dependency links were eventually added to Tropos [53], we did not consider them here. Dependencies can be treated as ++ links [68]. For example, depends ++ the g −−−→ g1 link in Figure 2.2(b) can be represented with a g1 −−−→ g link in Figure 2.2(c). For example, the BLE graph fragment in Figure 2.5 consists of eight intentions (modeled as goals), labeled Have Bike Lanes, Have Design, Build Bike Lanes, Cyclist Safety, Temporary Construction, Permanent Construction, Access to Parking, and Bike Lane Usage, and four relationships: and + (Have Design, Build Bike Lanes) −−−→ Have Bike Lanes, Permanent Construction −−−→ Access to Parking, or ++S (Temporary Const., Permanent Const.) −−−→ Build Bike Lanes, Have Bike Lanes −−−→ Cyclist Safety.

The speciﬁcation of the BLE graph fragment is hA, G, Ri where, A = {}, G = {hHave Bike Lanes, goali, hHave Design, goali, hBuild Bike Lanes, goali, hTemporary Construction, goali, hPermanent Construction, goali, hCyclist Safety, goali, hAccess to Parking, goali, hBike Lane Usage, goali}, and ++S R = {(Have Design, Build Bike Lanes) −−−→ Have Bike Lanes, Have Bike Lanes −−−→ Cyclist Safety, or (Temporary Construction, Permanent Construction) −−−→ Build Bike Lanes, + Permanent Construction −−−→ Access to Parking}

Figure 2.6: Speciﬁcation of the Bike Lanes example goal graph fragment.

2.2.2 Intention Evaluation

In order to evaluate the fulﬁllment (or satisfaction) of intentions, we deﬁne evidence pairs.

Deﬁnition 2.7 (Evidence Pair). Let a intention g ∈ G be given. An evidence pair is a pair (s, d) where s ∈ {F, P, ⊥} is the level of evidence for and d ∈ {F, P, ⊥} is the level of evidence against the fulﬁllment of g.

F [resp. P] means there is full [resp. partial] evidence for or against the fulfillment of an intention, and ⊥ represents null evidence. Together the evidence pairs form the lattice on ≥ (“more true”) shown in Figure 2.7, which is the universe of all evidence pairs E. Intentions can have one of five values: (Fully) Satisfied (F, ⊥), Partially Satisfied (P, ⊥), Partially Denied (⊥, P), (Fully) Denied (⊥, F), and None (⊥, ⊥). This combination produces four conflicting values (F, F), (F, P), (P, F), and (P, P). Table 2.5 lists the mappings between evidence pairs, and their symbol and visual syntax. Notice that we use the same symbol for non-conflicting values as iStar does, and use a simplified visual syntax for non-conflicting values. We use evidence pairs when giving a formal specification and use symbols when describing values in the text, for readability.

Deﬁnition 2.8 (Intention Evaluation). Let an intention g ∈ G be given. The evaluation of g, denoted by g , is a mapping G → E ∪ {⊥}. The evaluation of each intention is either an evidence pair or is set J K to a special value ⊥, meaning “no information”. Chapter 2. Preliminaries 21

Table 2.5: Legend of evidence pairs.

Evidence Visual Name Pair Symbol Syntax (Fully) Satisfied (F, ⊥) FS FS Partially Satisfied (P, ⊥) PS PS Partially Denied (⊥, P) PD PD (Fully) Denied (⊥, F) FD FD None (⊥, ⊥) N N (F, F) (F, F) (F, F) (F, P) (F, P) (F, P) Conflicting Values (P, F) (P, F) (P, F) (P, P) (P, P) (P, P)

For example, g = (F, ⊥) means there is full evidence for and no evidence against the fulﬁllment of J K g. Similarly, in the Bike Lanes example we can give an intention an evaluation by applying an evidence pair, such as Have Design = (⊥, F) to indicate that we do not have a design. J K We deﬁne four evidence predicates FS(g), PS(g), PD(g), and FD(g) over the evidence pairs. FS(g) ⇐⇒ ( g = (F, ∗)) J K PS(g) ⇐⇒ ( g = (F, ∗) ∨ (P, ∗)) J K PD(g) ⇐⇒ ( g = (∗, F) ∨ (∗, P)) J K FD(g) ⇐⇒ ( g = (∗, F)) J K This evaluation clearly guarantees the expected ground axioms from [52]: FS(g) =⇒ PS(g) FD(g) =⇒ PD(g)

2.2.3 Relationship Semantics and Propagation

We now describe the meaning of the relationships defined in Definition 2.6. N-ary intention relationships and or (g1, ..., gn) −−−→ g [resp. (g1, ..., gn) −−−→ g] mean that the fulfillment of g requires all [resp. only one] of no g1, ..., gn to be satisfied. (g1, ..., gn) −−−→ g means the absence of a relationship. The binary relationship +S ++S g1 −−−→ g [resp. g1 −−−→ g] means that if g1 is satisfied then there exists some [resp. full] evidence that +D ++D g is satisfied. g1 −−−→ g [resp. g1 −−−→ g] says that if g1 is denied then there exists some [resp. full] + ++ evidence that g is denied. g1 −−−→ g [resp. g1 −−−→ g] combines the +S and +D [resp. ++S and ++D] relationships to propagate some [resp. full] evidence for g when there is any evidence for g1. Similarly, -S --S g1 −−−→ g [resp. g1 −−−→ g] says that if g1 is satisfied then there exists some [resp. full] evidence that g is denied. The -D, --D, -, and -- relationships follow this same pattern and structure. In this thesis, evaluation of a graph hA, G, Ri is a process that assigns each intention g ∈ G an evidence pair either directly or via propagation. When propagation goes from source nodes to target nodes, it uses rules on lines 1–32 in Figure 2.8, and is called forward propagation. Backward propagation (lines 33-64 in Figure 2.8) propagates evidence from target intentions to source intentions and is a way Chapter 2. Preliminaries 22

(F, ⊥)

(F, P) (P, ⊥)

(F, F) (P, P) (⊥, ⊥)

(P, F) (⊥, P)

(⊥, F)

Figure 2.7: A lattice of elements in E.

goal relations relation Axioms  and V and V  If (g1, ..., gi, ..., gn) −−−−→ g then i FS(gi) ∨ 33 1 (g1, ..., gi, ..., gn) −−−−→ g ( FS(gi)) =⇒ FS(g)  i  or W V  If (g1, ..., gi, ..., gn) −−−−→ g then i FS(gi) ∨ 34 2 ( i PS(gi)) =⇒ PS(g)   ++S 3 V (FD(g ) =⇒ FD(g)) For every ri : gi −−−−→ g F S(gi) ∨ 35 i i FS(g) =⇒ ++ 4 V (PD(g ) =⇒ PD(g))  For every ri : gi −−−−→ g F S(gi) ∨ 36 i i  or V  For every r : g −−−−→--D g F D(g ) ∨ 37 5 (g1, ..., gi, ..., gn) −−−−→ g (FS(gi) =⇒ FS(g))  i i i i  -- V  For every ri : gi −−−−→ g F D(gi) 38 6 i(PS(gi) =⇒ PS(g)) 7 (V FD(g )) =⇒ FD(g)  and i i  If (g , ..., g , ..., g ) −−−−→ g then V PS(g ) ∨ 39 V  1 i n i i 8 ( i PD(gi)) =⇒ PD(g)  or W  If (g1, ..., gi, ..., gn) −−−−→ g then i PS(gi) ∨ 40 +S  9 g1 −−−−→ g P S(g1) =⇒ PS(g)  ++S  For every ri : gi −−−−→ g P S(gi) ∨ 41 -S  10 g1 −−−−→ g P S(g1) =⇒ PD(g)  +S  For every ri : gi −−−−→ g P S(gi) ∨ 42 ++S  ++ 11 g1 −−−−→ g F S(g1) =⇒ FS(g)  For every r : g −−−−→ g P S(g ) ∨ 43 PS(g) =⇒ i i i 12 PS(g ) =⇒ PS(g) + 1  For every ri : gi −−−−→ g P S(gi) ∨ 44 --S  13 g −−−−→ g F S(g ) =⇒ FD(g)  For every r : g −−−−→--D g P D(g ) ∨ 45 1 1  i i i  -D 14 PS(g1) =⇒ PD(g)  For every r : g −−−−→ g P D(g ) ∨ 46  i i i +D  -- 15 g1 −−−−→ g P D(g1) =⇒ PD(g)  For every ri : gi −−−−→ g P D(gi) ∨ 47  -D  - 16 g1 −−−−→ g P D(g1) =⇒ PS(g) For every ri : gi −−−−→ g P D(gi) 48 ++D  17 g −−−−→ g F D(g ) =⇒ FD(g) and W 1 1  If (g1, ..., gi, ..., gn) −−−−→ g then i FD(gi) ∨ 49  or 18 PD(g1) =⇒ PD(g)  If (g , ..., g , ..., g ) −−−−→ g then V FD(g ) ∨ 50  1 i n i i --D  ++D 19 g1 −−−−→ g F D(g1) =⇒ FS(g)  For every ri : gi −−−−→ g F D(gi) ∨ 51 FD(g) =⇒ ++ 20 PD(g1) =⇒ PS(g)  For every ri : gi −−−−→ g F D(gi) ∨ 52 +  --S 21 g1 −−−−→ g P S(g1) =⇒ PS(g)  For every r : g −−−−→ g F S(g ) ∨ 53  i i i  -- 22 PD(g1) =⇒ PD(g)  For every ri : gi −−−−→ g F S(gi) 54 - 23 g1 −−−−→ g P S(g1) =⇒ PD(g)  and  If (g , ..., g , ..., g ) −−−−→ g then W PD(g ) ∨ 55  1 i n i i 24 PD(g1) =⇒ PS(g)  or V ++  If (g1, ..., gi, ..., gn) −−−−→ g then i PD(gi) ∨ 56 25 g −−−−→ g F S(g ) =⇒ FS(g)  1 1  ++D  For every ri : gi −−−−→ g P D(gi) ∨ 57 26 PS(g1) =⇒ PS(g)   For every r : g −−−−→+D g P D(g ) ∨ 58 27 FD(g ) =⇒ FD(g)  i i i 1  ++  For every ri : gi −−−−→ g P D(gi) ∨ 59 28 PD(g1) =⇒ PD(g) PD(g) =⇒ + --  For every ri : gi −−−−→ g P D(gi) ∨ 60 29 g −−−−→ g F S(g ) =⇒ FD(g)  1 1  --S  For every ri : gi −−−−→ g P S(gi) ∨ 61 30 PS(g1) =⇒ PD(g)   -S  For every ri : gi −−−−→ g P S(gi) ∨ 62 31 FD(g1) =⇒ FS(g)   --  For every ri : gi −−−−→ g P S(gi) ∨ 63 32 PD(g1) =⇒ PS(g)   -  For every ri : gi −−−−→ g P S(gi) 64

(a) Forward propagation rules, adapted from [52]. (b) Backward propagation rules, adapted from [52].

Figure 2.8: Propagation rules applied to evidence predicates. Chapter 2. Preliminaries 23

Table 2.6: Forward propagation of satisﬁed and denied evidence pairs, adapted from [52].

and or (g1, ..., gi, ..., gn) −−−→ g (g1, ..., gi, ..., gn) −−−→ g

Sat(g) min{Sat(g1), ..., Sat(gi), ..., Sat(gn)} max{Sat(g1), ..., Sat(gi), ..., Sat(gn)}

Den(g) max{Den(g1), ..., Den(gi), ..., Den(gn)} min{Den(g1), ..., Den(gi), ..., Den(gn)} +S -S ++S --S g1 −−−→ g g1 −−−→ g g1 −−−→ g g1 −−−→ g

Sat(g) min{Sat(g1), P } ⊥ Sat(g1) ⊥

Den(g) ⊥ min{Sat(g1), P } ⊥ Sat(g1) +D -D ++D --D g1 −−−→ g g1 −−−→ g g1 −−−→ g g1 −−−→ g

Sat(g) ⊥ min{Den(g1), P } ⊥ Den(g1)

Den(g) min{Den(g1), P } ⊥ Den(g1) ⊥ + - ++ -- g1 −−−→ g g1 −−−→ g g1 −−−→ g g1 −−−→ g

Sat(g) min{Sat(g1), P } min{Den(g1), P } Sat(g1) Den(g1)

Den(g) min{Den(g1), P } min{Sat(g1), P } Den(g1) Sat(g1)

of obtaining values for target nodes. Backward propagation rules are derived directly from the forwards propagation rules, when all rules are taken together.

For example, the forward propagation of a +S relationship between g1 and g (see line 9 in Figure 2.8) means that if there is partial evidence for the fulfillment of intention g1, then there is partial evidence for the fulfillment of intention g. In a different example, if there is full evidence for the satisfaction of g (see lines 33-38 in Figure 2.8) then using backwards propagation at least one of the relations must have supplied this full evidence. If there is an and relation between (g1, ..., gn) and g (see line 33) then all source intentions g1, ..., gn must have full satisfaction evidence. Alternatively, if there is a ++S relationship between gi and g (see line 35) then the source intention gi must have full satisfaction evidence. In the Bike Lanes example, the forward propagation of the + relationship between Permanent Construction and Access to Parking means that if there is partial evidence for the fulfillment of intention Permanent Construction, then there is partial evidence for the fulfillment of intention Access to Parking. This + relationship also means that if there is partial evidence against the fulfillment of intention Permanent Construction, then there is partial evidence against the fulfillment of intention Access to Parking.

2.2.4 Propagation Algorithms

Prior to defining an evaluation over a goal graph, we introduce algorithms for propagating evidence pairs, including prior work [51, 52]. As mentioned above, when evidence pairs are propagated from source nodes to target nodes, this is called forward propagation. We first consider adapted version of forward propagation algorithm introduced by Giorgini et al. [52]. Recall that an evidence pair is a pair (s, d), where s ∈ {F, P, ⊥} is the level of evidence for and d ∈ {F, P, ⊥} is the level of evidence against the fulfillment of g, where F > P > ⊥. In the following algorithms, we use Sat(g) to refer to the s value of an evidence pair and Den(g) to refer to the d value of an evidence pair. The forward propagation rules are listed on lines 1–32 in Figure 2.8. To make the propagation algorithm easier to understand, in Table 2.6 we restate these rules in terms of (s,d) pairs (i.e., Sat(g) and Den(g)) of the source and target intentions. For example, the first row giving the Sat and Den rules for decomposition (i.e., and Chapter 2. Preliminaries 24

Algorithm 2.1 Update Label: Resolve Forward Propagation of Target Intention, Adapted From [52] 1: function Update Label(g, R, Old) . Old is the set of intentions and evidence pairs. Type: hg, g i. 2: (satg, deng) = Old.getEval(g) J K . Store old value. 3: for all r ∈ R do 4: if r.targetGoal == g then 5: satg = max(satg, Apply Rules Sat(g, r, Old)) 6: deng = max(deng, Apply Rules Den(g, r, Old))

7: return (satg, deng) . Return new evidence pair.

& or) directly connects to the rules on Lines 1–4 for and and Lines 5–8 for or.

UDATE LABLE Function (Algorithm 2.1). Suppose that a target has more than one incoming link. Using Table 2.6 is insuﬃcient because it does not specify how to combine values from multiple links. The Update Label function (see Algorithm 2.1) speciﬁes how the evidence pair for an intention g is updated based on the links for which g is the target intention. Update Label takes as input an intention g, a set of relationships R, and a set of tuples Old mapping intentions to evidence pairs, and returns a new evidence pair assignment for the input goal g. Line2 assigns the old evidence pair to satg and deng. Then, the main loop (Lines3–6) iterates through all relationships in the model to select only the relationships where g is the target node. For each of these incoming relationships, the algorithm invokes Apply Rules Sat and Apply Rules Den returning new sat and den values, respectively, for the selected relationship based on the rules in Table 2.6. After iterating over all the incoming relationships the updated sat and den values for the goal g are the maximum values (i.e., strongest evidence) that was propagated from one of the incoming links. For example, suppose Old contains the tuples hHave Bike Lanes,(⊥, ⊥)i, hHave Design,(F, ⊥)i, and hBuild Bike Lanes,(F, F)i. When Update Label is called for Have Bike Lanes, the returned value for Have Bike Lanes will be (P, F), based on the and relationship between them. In this dissertation, we give worst case runtimes in terms of O(n), where n is the number of intentions in a goal graph |G|. When an algorithm iterates over the set of relationships R, we approximate the runtime to be n2, because the number of edges in the worst case is quadratic in the number of nodes (i.e., |R| is n2)[33]. Thus, Update Label has a worst case runtime of O(n2) because it iterates |R| times to check each relationship in the model. We use the Update Label function in the next two procedures to propagate evidence pairs throughout the model.

LABEL GRAPH Function (Algorithm 2.2). Giorgini et al. speciﬁed the Label Graph function presented in Algorithm 2.2[52]. The Label Graph function forward propagates evidence pairs throughout the goal graph, which allow users to complete forward analysis and ask “what if” questions. The function takes as input a goal graph hA, G, Ri, and a set Initial (where |Initial| = |G|). Initial contains tuples that map each intention to evidence pairs in E. Label Graph returns Current which is an updated mapping for evidence pair assignments for each of the intentions in G. After initialization on Line2, each iteration of the main loop Lines3–7 makes a copy of the assignments ( Old) and then for each intention in G it updates the evidence pairs for all incoming relationships by calling Update Label on Line6. The loop completes when a ﬁx-point is found where no future updates can be made to the goal graph (i.e., Current = Old). For example, to perform forward analysis on the goal graph in Figure 1.2, to answer the question, Chapter 2. Preliminaries 25

Algorithm 2.2 Label Graph: Forward Graph Evaluation from Leaf Intentions, Adapted From [52] 1: function Label Graph(hA, G, Ri, Initial) 2: Current = Initial . Initialize Current 3: do 4: Old = Current 5: for all g ∈ G do 6: Current(g) = Update Label(g, R, Old) . See Algorithm 2.1. 7: while Current 6= Old . Iterates until ﬁx-point is reached. 8: return Current

“what is the impact on Cyclist Safety if we choose Permanent Construction instead of Temporary Construction?” Algorithm 2.2 is called with the goal graph in Figure 1.2 (specified in Section 2.2.1) and the following set for Initial. Initial = {hBike Lane Usage,(P, ⊥)i, hHave Design,(F, ⊥)i, hPermanent Construction,(F, ⊥)i, hTemporary Construction,(⊥, F)i, hAccess to Parking,(⊥, ⊥)i, hBuild Bike Lanes,(⊥, ⊥)i, hCyclist Safety,(⊥, ⊥)i, hHave Bike Lanes,(⊥, ⊥)i}. The following value for Complete is returned, where a fix-point is reached and Cyclist Safety is satisfied. Complete = {hBike Lane Usage,(P, ⊥)i, hHave Design,(F, ⊥)i, hPermanent Construction,(F, ⊥)i, hTemporary Construction,(⊥, F)i, hAccess to Parking,(P, ⊥)i, hBuild Bike Lanes,(F, ⊥)i, hCyclist Safety,(F, ⊥)i, hHave Bike Lanes,(F, ⊥)i}. So we have answered the questions that if Permanent Construction is chosen instead of Temporary Construc- tion, it results in the satisfaction (F, ⊥) of Cyclist Safety. Giorgini et al. proved soundness and completeness of the Label Graph function (in conjunction with the Update Label function, see Algorithm 2.1), by showing that the maximum evidence values for each of s and d would be deduced from Initial [52]. Giorgini et al. also proved that the main loop terminates after at most 6|G| + 1 iterations [52]. Within the main loop, we iterate over the intentions (i.e., |G| iterations) on Line5, and call Update Label. Since Update Label has a worst case runtime of O(n2), the runtime of Update Label is O(n4).

2.2.5 Goal Graph Evaluation

As mentioned above, in evaluating a goal graph, we assigned evidence pairs either directly or via propagation. In order to evaluate a goal graph with the propagation rules, we must first define conflicting values and introduce how stakeholders directly assign evidence pairs. As mentioned above the universe of all evidence pairs E contains four conflicting evidence pairs: (F, F), (F, P), (P, F), and (P, P). In evaluating a goal graph, we may want to avoid these conflicting values where possible. We specify this avoidance with conflict levels.

Definition 2.9 (Conflict Levels). The set Weak Conflict = {(F, P), (F, F), (P, P), (P, F)} represents conflicting evidence pairs, i.e., pairs of values that should not be assigned to an intention. In addition, we define subsets of Weak Conflict which are often meaningful: Strong Conflict = {(F, F)}, Medium Conflict = {(F, P), (F, F), (P, F)}, and None Conflict = {}.

Medium Conflict avoidance requires that an intention cannot be fully satisfiable and (fully or) partially deniable, and vice versa. Strong Conflict avoidance requires that an intention cannot be fully satisfiable and fully deniable. In ideal scenarios, stakeholders may avoid all conflicting values (i.e., Chapter 2. Preliminaries 26

Weak Conflict), but this is not always possible. Strong Conflicts should be avoided because they are rarely helpful in decision making. When an intention has full evidence for and against its fulfillment, this is likely due to contribution links, and stakeholders should explore developing the model further. Part of the GORE approach is making tradeoff decisions when the best alternative is not obvious. For this reason, Weak Conflict and Medium Conflict are generally tolerated because not all soft-goals or non-functional requirements can be satisfied by any alternative.

Goal evaluations can be assigned by users prior to evaluating the graph as a whole. Goal evaluations allow stakeholders to assign desired evidence pairs to intentions for analysis, as well as evidence pair assignments that are known to be true prior to analysis.

Deﬁnition 2.10 (User Evaluation). Let a goal graph hA, G, Ri be given. A set of user evaluations UEval = {hg, ei | g ∈ G ∧ e ∈ E} is the user’s assignment of evidence pairs to intentions.

For example, a set of user evaluations for the Bike Lanes example could be {hHave Bike Lanes,(F, ⊥)i, hTemporary Construction,(F, ⊥)i, hAccess to Parking,(⊥, P)i}. With these inputs, we can now deﬁne evaluation over a goal graph.

Deﬁnition 2.11 (Goal Graph Evaluation). Let a well-formed goal graph hA, G, Ri, a set of user evaluations UEval, and a conﬂict level CFLevel be given. A complete evaluation of hA, G, Ri is the total mapping G → E resulting from the repeated assignment of evidence pairs to input intentions and application of the propagation rules in Figure 2.8 to all relationships in R, given UEval, such that ∀g ∈ G : g 6∈ CFLevel ∧ g 6= ⊥ ∧ ( ∀hg, ei ∈ UEval : e = g ). When this is not possible, the J K J K J K evaluation of the goal graph is said to be infeasible.

Infeasible evaluations occur when the evaluation is over-constrained by one or more of the goal graph relationships, the user evaluations, and the conﬂict level. Consider the relationship and (Have Design, Build Bike Lanes) −−−→ Have Bike Lanes from the Bike Lanes example. If stakeholders add the following two user evaluations: {hHave Design,(⊥, F)i, hHave Bike Lanes,(F, ⊥)i}, the resulting goal graph evaluation is infeasible, because it is not possible for Have Bike Lanes to be satisﬁed (i.e., have the evidence pair (F, ⊥)) while Have Design is denied (i.e., with the evidence pair (⊥, F)) as this violates the propagation rules of the and relationship.

2.2.6 Evaluation Algorithms

In this section, we present the algorithm for ﬁnding complete evaluations of a goal graph (satisfying Def- inition 2.11) and two helper functions used in the algorithm.

PROPAGATE Function (Algorithm 2.3). We begin by looking at the Propagate function (see Al- gorithm 2.3). This function propagates values from source nodes to target nodes. In Section 2.2.4, we introduced the Label Graph function (see Algorithm 2.3), which propagates values throughout a goal graph until a fix-point is achieved. Propagate differs from Label Graph in that instead of having the input Initial (where |Initial| = |G|), Propagate takes as input two sets, uGoals and aGoals, to represent unassigned (i.e., having the value ⊥) and assigned (i.e., having a value in E) values, respectively. Each element of aGoals is of the form hg, g i and may be specified by the user. Each element of uGoals J K is of the form hg, ∅i, where the second element in the tuple is a set of possible evidence pairs in E, with Chapter 2. Preliminaries 27

Algorithm 2.3 Propagate: Forward Graph Evaluations for Partially Evaluated Goal Graphs 1: function Propagate(R, aGoals, uGoals) . Propagates values where possible. 2: do 3: size = |aGoals| . Store the size of the aGoal list to see if it changes. 4: for all r ∈ R do 5: tGoal = r.targetGoal . Get target intention. 6: if aGoals.getGoal(tGoal) 6= null then continue . Move to next relationship, this intention already has a value. 7: sGoals = r.sourceGoalSet . Get source intentions. 8: if sGoals \ aGoals.goals = ∅ then . All source intentions are assigned. 9: tGoalEval = Update Label(tGoal, {r}, aGoals) . See Algorithm 2.1. 10: if tGoalEval ∈ uGoals.getPossibleEvidence(tGoal) then . Check assignment. 11: aGoals.add(tGoal, tGoalEval) 12: uGoals.remove(tGoal) 13: while |aGoals|= 6 size 14: return (aGoals, uGoals)

the possible values varying depending on the given conflict level. Propagate returns an updated copy of aGoals and uGoals. Similar to Label Graph, the main loop (see Lines2–13) iterates until a fix- point is found, where no additional evidence pairs have been assigned to intentions (i.e., |aGoals| = size, on Line 13). The updated sets are returned on Line 14. The inner loop (see Lines4–12) evaluates each relationship r ∈ R. If the target of r is already assigned, then the relationship is skipped (see Line6). If the target of r is not already assigned and all of the source intentions of r are assigned (Line8), then Update Label is called to assign an evidence pair for the target intention (see Algorithm 2.1). Finally, the target intention is removed from uGoals and added to aGoals with the assigned evidence pair (Lines 11 and 12). A value is only assigned to a target intention when all source intentions have as assigned value in E. For example, consider the Bike Lanes example fragment in Figure 1.2 (with R specified in Section 2.2.1) and the following values for aGoals and uGoals: aGoals = {hBike Lane Usage,(P, ⊥)i, hPermanent Construction,(F, ⊥)i, hTemporary Construction,(⊥, F)i}, uGoals = {hHave Design, {(F, ⊥), (P, ⊥), (⊥, P), (⊥, F), (⊥, ⊥)}i, hAccess to Parking, {(F, ⊥), (P, ⊥), (⊥, P), (⊥, F), (⊥, ⊥)}i, hBuild Bike Lanes, {(F, ⊥), (P, ⊥), (⊥, P), (⊥, F), (⊥, ⊥)}i, hCyclist Safety, {(F, ⊥), (P, ⊥), (⊥, P), (⊥, F), (⊥, ⊥)}i, hHave Bike Lanes, {(F, ⊥), (P, ⊥), (⊥, P), (⊥, F), (⊥, ⊥)}i}, are used as inputs to Algorithm 2.3. The resulting output value for aGoals is: aGoals = {hBike Lane Usage,(P, ⊥)i, hPermanent Construction,(F, ⊥)i, hTemporary Construction,(⊥, F)i, hAccess to Parking,(P, ⊥)i, hBuild Bike Lanes,(F, ⊥)i}. Have Bike Lanes and Cyclist Safety are not assigned values because Have Design does not have a value and one cannot be generated through forward analysis. If we assume that Update Label (see Algorithm 2.1) is correct, then each assignment made by the Propagate function is also correct because Propagate assigns a target value only when all source intentions are assigned evidence pairs. Since this algorithm does not deal with a complete evaluation of the goal graph, we do not make claims about the correctness of every assignment in the graph, and use Check Prop (see Algorithm 2.4) to evaluate the graph as a whole. Chapter 2. Preliminaries 28

Algorithm 2.4 Check Prop: Checks Backward Propagation Rules for All Assigned Relationships 1: function Check Prop(R, aGoals) . Checks backwards rules for evidence pair assignments. 2: for all t ∈ aGoals do 3: linkEvalSet = ∅ . Set of pairs of link type and source intention evidence pair set. Type: htype, { g1 ··· gn }i 4: rSet = R.getTargetRel(t.goal) . Get relationship set that has this intentionJ K as aJ target.K 5: for all r ∈ rSet do 6: sEvals = ∅ . Holds the set of evidence pairs for source intentions. 7: for all s ∈ r.sourceGoalSet do 8: if aGoals.getGoal(s) 6= null then . No source intentions have assignments. 9: sEvals.add(aGoals.getEval(s)) . Add evidence pair to set. 10: if sEvals 6= ∅ then 11: linkEvalSet.add(r.type, sEvals) 12: if linkEvalSet == ∅ then continue 13: if ¬(Back Sat(t.eval, linkEvalSet)) then return false . See rules in Figure 2.8(b). 14: return true

The runtime of the Propagate algorithm depends on the depth of the goal graph. If we assume that the graph consists of a path of k links, and all leaf-level intentions are elements of aGoals, then the main loop (see Lines2–13) will iterate k times in the worst case, and each iteration will assign a single new evidence pair to a target intention on the path of length k. If the graph is a chain, then k = n in the worst case. Recall that the runtime of Update Label is O(n2). The inner loop of Propagate (see Lines4–12) always iterates |R| times (i.e., order n2); thus, the worst case runtime of Propagate is O(n5). Notice that Propagate has a higher complexity than Update Label. Although this is not ideal, Propagate is called by Algorithm 2.5, which requires that Propagate diﬀerentiate between None (⊥, ⊥) and unassigned intentions ⊥, which Update Label cannot do.

CHECK PROP Function (Algorithm 2.4). The Check Prop function (see Algorithm 2.4) inspects whether any intention evaluations in a goal graph violate the backward propagation rules for relationships (see Figure 2.8(b)). This function is used to verify that assignments made by Propagate (see Algo- rithm 2.3) satisfy all propagation rules. The inputs to Check Prop are R: the set of relationships in the goal graph, and aGoals: a set of tuples that map intentions to evidence pairs (hg, g i). The main J K loop (see Lines2–2) iterates over each intention in aGoals and check the relationships where the goal is a target node. After initialization, the relationship loop (see Lines5–11) iterates over each relationship where t is the target goal. For each relationship, the function loops over the source intentions to determine if they are assigned (see Lines7–9), meaning that they are elements of aGoals. Evidence pairs for source goal are stored in sEvals. If the source goal has an assignment sEvals 6= ∅, then the relationship is added to linkEvalSet to be checked. When all relationships have been added to linkEvalSet, Back Sat is called on Line 13, which checks the assignments against the rule in Figure 2.8(b). If a violation is found, Check Prop returns false. If an intention has no incoming relationships, or if the source intentions are not assigned, then the goal is skipped (see Line 12). Once all intentions have been checked, no violations have been discovered and Check Prop returns true. For example, Check Prop would return true for the output of the Propagate example discussed above: aGoals = {hBike Lane Usage,(P, ⊥)i, hPermanent Construction,(F, ⊥)i, hTemporary Construction,(⊥, F)i, hAccess to Parking,(P, ⊥)i, hBuild Bike Lanes,(F, ⊥)i}. Chapter 2. Preliminaries 29

Algorithm 2.5 Compute Goal Graph Evaluation Input: Goal Graph hA, G, Ri . Satisfying Definition 2.6. Set of User Evaluations UEval . Satisfying Definition 2.10. Set of Conflicting Evidence Pairs CF Evidence . Satisfying Definition 2.9. Output: Evaluations of the graph intentions (satisfying Definition 2.11) or null indicating infeasibility.

1: aGoals = ∅ . Set of assigned intentions and evidence pairs. Type: hg, g i. 2: uGoals = ∅ . Set of unassigned intentions and possible evidence pairs. Type: hg,J{}iK . 3: for all g ∈ G do 4: if g ∈ UEval then 5: if UEval(g) ∈ CF Evidence then return null 6: aGoals.add(hg, UEval(g)i) 7: else 8: uGoals.add(hg, E\ CF Evidencei) . Add the set of non-conﬂicting evidence pairs. 9: return Solve(aGoals, uGoals, R)

10: function Solve(aGoals, uGoals, R) . Recursive procedure to pick values and check. 11: if uGoals = {} then 12: return aGoals 13: oldAGoals = aGoals, oldUGoals = uGoals . Create copies for backtracking. 14: curGoal = uGoals.head.goal . Get values at head of uGoals. 15: curEvals = uGoals.head.evals 16: for all val ∈ curEvals do . Iterate over possible evaluations. 17: aGoals.add(curGoal, val) . Make assignment. 18: uGoals.remove(curGoal) 19: (aGoals, uGoals) = Propagate(R, aGoals, uGoals)) 20: if ¬Check Prop(R, aGoals) then goto Line 23 21: result = Solve(aGoals, uGoals, R) . Call SOLVE to select the next assignment. 22: if result 6= null then return result . Return successful result. 23: aGoals = oldAGoals . Backtracking: restore previous assignments. 24: uGoals = oldUGoals 25: return null . If no evaluations remain, the graph is infeasible.

However, only the and relationship for Build Bike Lanes and the + relationship for Access to Parking would have been checked by Back Sat on Line 13. The Check Prop function is trivially correct because it simply iterates over all intention relationship combinations and checks the rules in Figure 2.8(b). The worst case run-time of this algorithm occurs if every intention in the graph has an assignment in aGoals, meaning |aGoals| = |G|. This means that Back Sat (which is O(1)) will be called |G| times in the worst case, resulting in a runtime of O(n) for Check Prop.

Goal Graph Evaluation Algorithm (Algorithm 2.5). Finally, we describe Algorithm 2.5, a brute force algorithm that returns a complete goal graph evaluation (satisfying Definition 2.11) or null indicating infeasibility. Algorithm 2.5 can be used in backwards analysis to generate scenarios where desired evidence pairs are achieved. Algorithm 2.5 has three inputs: a goal graph (satisfying Definition 2.6), a set of user evaluations (satisfying Definition 2.10), and a set of conflicting evidence pairs (satisfying Definition 2.9). Algorithm 2.5 consists of two parts: initialization of the inputs (see Lines1–8) and the Solve function (see Lines 10– Chapter 2. Preliminaries 30

Figure 2.9: BLE-Fragment: prior to evaluation. Figure 2.10: BLE-Fragment: after evaluation.

25). Solve is a recursive function that incrementally assigns evidence pairs. When Solve is first called on Line9, it contains only the user evaluations in aGoals, and the remainder of the intentions in G are stored in uGoals with non-conflicting evidence pairs. For each iteration of the main loop (see Lines 16–24), Solve considers one additional assigned evidence pair. The new assignment, val for curGoal, is added to aGoals and removed from uGoals. Then the algorithm attempts to assign additional evidence pairs to intentions by calling the Propagate function on Line 19 (see Algorithm 2.3 for the Propagate function), and then checks that the assignment was correct with the Check Prop function on Line 20 (see Algorithm 2.4 for the Check Prop function). If Check Prop returns false (meaning the graph is inconsistent), then the algorithm jumps to Line 23 to backtrack. If all the checks return true then a recursive call to Solve is made with the additional assignment on Line 21. If Solve returns with a set of aGoals, then it was successful and result is returned; otherwise, backtracking is required. Solve backtracks by reverting aGoals and uGoals back to their old values and begins the next iteration of the main loop, where the next val is selected. If the main loop completes and all elements in curEvals has been tested, then no solution exists and Solve returns null indicating infeasibility. Solve exhaustively checks every evidence pair for every unassigned intention at every time point, in the case of infeasibility. If one were to implement Algorithm 2.5, there are many ways of improving it, including caching and dynamic programming to store intermediate results. Illustration of Algorithm 2.5. We describe the evaluation of the goal graph in Figure 2.5 step-by- step with Algorithm 2.5 (see Figure 2.9 and Figure 2.10 for illustration). In this example, the inputs to Algorithm 2.5 are a goal graph hA, G, Ri as specified in Figure 2.6, a set of user evaluations UEval = {hHave Bike Lanes,(F, ⊥)i, hTemporary Construction,(F, ⊥)i, hAccess to Parking,(⊥, P)i}, and Weak Conflict as the conflict avoidance level CF Evidence = {(F, P), (F, F), (P, P), (P, F)}. Fig- ure 2.9 shows the goal graph with these user evaluations prior to Algorithm 2.5. Chapter 2. Preliminaries 31

After initialization on Lines1–8, aGoals and uGoals have the following values when the ﬁrst call to Solve is made on Line9:

aGoals = {hHave Bike Lanes,(F, ⊥)i, hTemporary Construction,(F, ⊥)i, hAccess to Parking,(⊥, P)i}, uGoals = {hHave Design, {(F, ⊥), (⊥, F), (P, ⊥), (⊥, P), (⊥, ⊥)}i, hCyclist Safety, {(F, ⊥), (⊥, F), (P, ⊥), (⊥, P), (⊥, ⊥)}i, hPermanent Construction, {(F, ⊥), (⊥, F), (P, ⊥), (⊥, P), (⊥, ⊥)}i, hBuild Bike Lanes, {(F, ⊥), (⊥, F), (P, ⊥), (⊥, P), (⊥, ⊥)}i, hBike Lane Usage, {(F, ⊥), (⊥, F), (P, ⊥), (⊥, P), (⊥, ⊥)}i}

In Table 2.7, we illustrate how the values of aGoals and uGoals are updated for each intention in uGoals by the Solve function. Have Bike Lanes, Temporary Construction, and Access to Parking are not included in Table 2.7 because they are user evaluations, and their assignments remain fixed. Row 1 (see Table 2.7) shows the values at the start when Solve is initially called. Since uGoals is not empty (Line 11), Solve selects the first element in uGoals, which is Have Design, and tries the first value (F, ⊥). Row 2 shows the values after this assignment and before an propagation on Line Line 19. The call to the Propagate function assigned Cyclist Safety the (F, ⊥) evidence pair based on the ++S relationship between Have Bike Lanes and Cyclist Safety (see Row 3). With these values, the second call to Solve is made. At this point, the head of uGoals is Permanent Construction and the first value (F, ⊥) is assigned (see Row 4). Next, Check Prop on Line 20 returns false because the assignment of (F, ⊥) to Permanent Construction violates the + relationship with Access to Parking, where Access to Parking is (⊥, P). Backtrack- ing replaces the values of aGoals and uGoals on Lines 23 and 24 (see Row 5). On the second iteration of the main loop, (⊥, F) is assigned to Permanent Construction (see Row 6). This assignment is successful and results in the assignment of (F, ⊥) to Build Bike Lanes when Propagate is called (see Row 7). The Solve function is recursively called again (for the third time), and this time Bike Lane Usage is assigned (F, ⊥), the first value in the curEvals list. As all intentions have been assigned an evidence pair, any call to Propagate does not result in any propagation. Finally, Solve function is recursively called a forth and final time (see Row 9). This time, uGoals is an empty list, so aGoals is returned on Line 12. aGoals is then returned through each level of recursion on Line 22, and finally returning from the main function on Line 10. The output of Algorithm 2.5 is the set of aGoals, which is a complete goal graph evaluation, satisfying Definition 2.11, as visualized in Figure 2.10. aGoals = {hHave Bike Lanes,(F, ⊥)i, hTemporary Construction,(F, ⊥)i, hAccess to Parking,(⊥, P)i, hHave Design,(F, ⊥)i, hCyclist Safety,(F, ⊥)i, hPermanent Construction,(⊥, F)i, hBuild Bike Lanes,(F, ⊥)i, hBike Lane Usage,(F, ⊥)i}

Algorithm 2.5 updates the graph each time an intention is added to aGoals on Line 17. The maximum size of curEvals is |E| = 9; each intention can have at most nine possible evidence pair assignments. In the worst case, the goal graph evaluation is infeasible, and Solve will try every combination of evidence pairs for every intention, changing the graph assignment 9|G| times, before returning null. This assumes that no additional assignments are made by calling the Propagate function. For each assignment, Propagate (O(n5)) and Check Prop (O(n)) are called. The worst case runtime of Algorithm 2.5 is O(9n ∗ n5), which is dominated by the exponential function O(9n). This algorithm builds on the backwards propagation work of Sebastiani et al. [134], where they proved soundness and completeness of the backwards propagation rules in Figure 2.8(b) using the rules in Figure 2.8(a). Their work included cyclic graphs but for the purpose of backwards propagation, cycles Chapter 2. Preliminaries 32 ), ), ), ), ), ), ), ), ), ), ), ), ), ), F F F F F F F P P P P P P P , , , , , , , , , , , , , , ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ } } } } } } } ), ( ), ( ), ( ), ( ), ( ), ( ), ( ) ) ) ) ) ) ) ), ( ), ( ), ( ), ( ), ( ), ( ), ( ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ , , , , , , , ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ , , , , , , , , , , , , , , F F F F F F F ( ( ( ( ( ( ( ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ P P P P P P P ( ( ( ( ( ( ( ( ( ( ( ( ( ( uGoals { { { { { { { ) ) Bike Lane Usage ⊥ ⊥ , , F F ( ( aGoals ), ), ), ), ), ), ), ), ), ), ), ), F F F F F F P P P P P P , , , , , , , , , , , , ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ } } } } } } ), ( ), ( ), ( ), ( ), ( ), ( ) ) ) ) ) ) ), ( ), ( ), ( ), ( ), ( ), ( ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ , , , , , , ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ , , , , , , , , , , , , F F F F F F ( ( ( ( ( ( ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ P P P P P P ( ( ( ( ( ( ( ( ( ( ( ( uGoals { { { { { { ) ) ) Build Bike Lanes ⊥ ⊥ ⊥ , , , F F F ( ( ( aGoals ), ), ), ), ), ), ), ), F F F F P P P P from Line 9 . , , , , , , , , ) during all of Algorithm 2.5 and are excluded here. ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ P , ⊥ } } } } ), ( ), ( ), ( ), ( ) ) ) ) ), ( ), ( ), ( ), ( Solve ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ , , , , = ( ⊥ ⊥ ⊥ ⊥ , , , , , , , , F F F F ( ( ( ( P ⊥ P ⊥ P ⊥ P ⊥ uGoals { ( ( { ( ( { ( ( { ( ( ) ) ) ) ) F F F F ⊥ , , , , , Permanent Construction F ⊥ ⊥ ⊥ ⊥ ( ( ( ( ( aGoals Access to Parking ), ), ), ), F F P P , , , , ⊥ ⊥ ⊥ ⊥ ), and } } ), ( ), ( ) ) ⊥ ), ( ), ( ⊥ ⊥ , ⊥ ⊥ , , ⊥ ⊥ F , , , , F F . Where S1 is the first call to ( ( ⊥ ⊥ P P ( ( ( ( uGoals { { = ( Cyclist Safety ) ) ) ) ) ) ) Solve ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ , , , , , , , F F F F F F F ( ( ( ( ( ( ( aGoals ), ), F P , , ⊥ ⊥ } ), ( ) ), ( ⊥ Temporary Construction ⊥ , ⊥ , , F ), ( P ⊥ ( ( uGoals { ⊥ , Have Design F ) ) ) ) ) ) ) ) ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ = ( , , , , , , , , F F F F F F F F ( ( ( ( ( ( ( ( aGoals th rd nd Solve Table 2.7: Evaluations for initially unassigned intentions throughout Algorithm 2.5 for the Bike Lanes example goal graph fragment. Have Bike Lanes Note 1. (S1)Call Initial to (Line 9 ) 2. (S1)Line Before 19 3.Line (S1)19 /2 After Call on Line 21 4. (S2)Line Before 19 5.Line (S2)19 / After Back- track Lines 23 – 24 6.fore (S2) Line (Attempt 2) Be- 19 7.Line (S2)19 /3 After Call on Line 21 8. (S3)Line Before 19 9.Line (S3)/419 After Call on Line 21 Result Returned at Line 22 Note: (SX) Indicates the depth of the recursive calls to Chapter 2. Preliminaries 33 were converted to sub-DAGs, encompassing Definition 2.6. We used the same conflict avoidance levels (see Definition 2.9) and similar representations for user inputs (see Definition 4.2). In Section 4.1.3, we extend Algorithm 2.5 to perform reasoning over time. We will discuss runtime and correctness of the timed algorithm in detail; thus, we only briefly mention runtime here. We did not implement Algorithm 2.5 due to the exponential runtime. The intractable nature of the problem of goal graph evaluations motivates the need for more advanced searched-based techniques (i.e., constraint satisfaction problems (CSP) solvers), see Section C.1.

2.3 Summary

In this chapter, we introduced relevant background on iStar and Tropos. Using the Waste Management example, we illustrated the syntax of iStar and showed how evaluation labels can be propagated through forward and backward analysis. As we extended Tropos in this dissertation, we gave a more thorough presentation of the formal syntax and semantics of Tropos, illustrating the concepts with the Bike Lanes example. We also presented and illustrated algorithms for propagation of evidence pairs as well as evaluating a goal graph as a whole in Tropos. Part II

Evolving Intentions Framework

34 Chapter 3

Modeling Evolving Intentions

In this chapter, we present the modeling portion of the Evolving Intentions framework. In Chapter1, we argued that goal models without explicit modeling of time may lead to incorrect results because previous approaches do not account for variations in intentions’ satisfaction. To address these challenges, we propose providing analysis to allow stakeholders to consider alternatives over time. In this chapter, we describe how to specify and model changes in goal models by deﬁning functions for Evolving Intentions and Evolving Relationships as well as constraints between functions. With these functions we can specify an Evolving Goal Model. We begin by giving a complete presentation of the Evolving Intentions framework in Tropos and then present an example of how it is described in iStar. See Chapter2 for an overview of iStar and Tropos. In the next chapter, we describe how to reason with an evolving goal model.

Contributions

This chapter makes the following contributions. • We present a formal approach to modeling with evolution in Tropos using the Bike Lanes example. • We establish how time is represented in the Evolving Intentions framework. • We deﬁne general functions for Evolving Intentions over one or more time periods. • We also specify how relationships within a model can evolve over time. • We give a deﬁnition for an Evolving Goal Graph. • We describe how stakeholders can use Evolving Intentions to specify changes in iStar using the Waste Management example.

Organization

The remainder of this chapter is organized as follows: Section 3.1 gives the complete formalization of the Evolving Intentions framework in Tropos. Section 3.2 describes how stakeholders can use the Evolving Intentions framework in iStar. Section 3.3 provides a summary of these contributions.

35 Chapter 3. Modeling Evolving Intentions 36

3.1 Modeling in Tropos

In this section, we describe how we extended deﬁnitions in Section 2.2 and the BLE model fragment in Figure 2.5 to deﬁne the evolution of goal graphs over time.

3.1.1 Time Points and Intention Evaluations

In this work, we use discrete time where a tick is a minimal distance between two observable events. Time is measured in ticks from a user-deﬁned notion of beginning (zero) to a user-deﬁned notion of end (maxT ime). The universe of time is T = {t | t ∈ N0 ∧ t ∈ [0, maxT ime] ∨ t ∈ {⊥}}.

ticks 0 10 maxT ime

t0 t1 t2 t3 t4 0 6 8 12 17 t t timepoints 5 6 ⊥ ⊥

Figure 3.1: Mapping of ticks to time points.

Time Points (of Interest) are a universe of timed variables P ⊆ T ∪ {maxT ime + 1}. We evaluate goal graphs over sequences of time points. Figure 3.1 illustrates this mapping between ticks and time points. The top line shows a sequence of ticks starting at zero. The boxes underneath represent the time points. Time points may be represented solely by their symbolic constants or be deﬁned relative to other time points. t5 and t6 do not have time assignments. Note that we include {maxT ime + 1} in P with the rationale that we will be deﬁning intervals over time points that are open on the right and we want to make sure that maxT ime can be included.

We use t0 = 0 to denote the initial time value, and tend = maxT ime + 1 to denote the end of all open intervals where the ﬁnal value could be maxT ime.

Deﬁnition 3.1 (Intention Evaluation at a Time Point). Let an intention g ∈ G and a time point t ∈ P be given. The evaluation of g at t, denoted by g[t] , is a mapping G × P → E ∪ {⊥}. J K 3.1.2 Evolving Functions for Intentions

In this section, we introduce functions that describe how the evaluation of an intention changes over time. Atomic Functions. Between two time points, the evaluation of an intention can become more true, or Increase; become more false, or Decrease; remain Constant; or change randomly, displaying a Stochastic pattern. We will now deﬁne diﬀerent atomic functions over an interval of time points [start, stop) where start, stop ∈ P , given a special value x denoting a reference evidence pair.

Definition 3.2 (Atomic Functions). Let an intention g ∈ G, a set of time points P , a reference evidence pair x ∈ E ∧ x 6∈ Weak Conflict, and time points start, stop ∈ P (where start < stop) be given. An atomic function defines how the evaluation of g changes over the time points in P within an interval [start, stop), based on four types. Chapter 3. Modeling Evolving Intentions 37

Table 3.1: Time points associated with the Bike Lanes example. The following time points are used to describe formal concepts and explain the Bike Lanes example. In this example, maxT ime is assigned arbitrarily to be 50.

Introduced Symbol Description Absolute in Section Value

3.1.1 t0 Start of analysis. 0

3.1.1 tend maxT ime is the ﬁnal point of analysis. This time point ensures 51 that maxT ime can be selected as a point in the last interval of each evolving function.

3.1.2 tuse Marks the end of the Constant (⊥, F) period for Bike Lane 10 Usage, which is the time when the construction of the bike lanes is ﬁnished. Also marks the beginning of the repeating period, and the ﬁrst Stochastic function.

3.1.2 tspring Marks the beginning of spring and the transition to an In- ⊥ crease function in Bike Lane Usage.

3.1.2 tsummer Marks the beginning of summer and the transition to a Con- ⊥ stant function at (F, ⊥) in Bike Lane Usage.

3.1.2 tfall Marks the beginning of fall and the transition to a Decrease ⊥ function in Bike Lane Usage.

3.1.2 tdesign The time when Have Design became satisﬁed and a design has ⊥ been created.

3.1.3 tBLOpen The time when the bike lanes opened and the end of the neg- 10 ative impact of Permanent Construction on Access to Parking.

3.1.3 tdecision The time when a decision between Permanent Construction and 5 Temporary Construction is made.

3.1.4 ta0 Graph constraint time point. 7

3.1.4 ta1 Graph constraint time point. 14

3.1.4 ta2 Graph constraint time point. 34

(a) Constant says that the evidence pair assignment of an intention g remains equal to x. ∀ ti ∈

P : ti ∈ [start, stop) =⇒ g[ti] = x J K (b) Increase says changes in the evidence pair assignment becomes “more true”, up to a maximum value x as time progresses. ∀ ti, tj ∈ P : ti, tj ∈ [start, stop) ∧ ti < tj =⇒ g[ti] ≤ g[tj] ≤ x J K J K (c) Decrease says changes in the evidence pair assignment becomes “less true”, down to a minimum value x as time progresses. ∀ ti, tj ∈ P : ti, tj ∈ [start, stop) ∧ ti < tj =⇒ g[ti] ≥ g[tj] ≥ x J K J K (d) Stochastic says changes in the evidence pair assignment are stochastic or random, but do not include conﬂicting values. An atomic function is the tuple htype, x, start, stopi, that can be associated with an intention, where type is one of Constant, Increase, Decrease or Stochastic.

Generalized Function. When evaluating intentions, modelers can use atomic functions on their own or can create compound functions by deﬁning an ordered list of atomic functions over an ordered set of time intervals.

Deﬁnition 3.3 (Disjoint Neighbouring Intervals). Intervals [t1, t2), [t3, t4), [t5, t6)... are disjoint neighbouring intervals if t2 = t3 ∧ t4 = t5. Chapter 3. Modeling Evolving Intentions 38

(F, ⊥)

(⊥, F) t t t t t 0 summer spring fall end

Figure 3.2: Sequence of atomic functions over time points t0...tend.

Deﬁnition 3.4 (Compound Function). Let a sequence of atomic functions A be given, where ai ∈ A is the tuple htype, x, start, stopi and ∀ai ∈ A : ai(stop) = ai+1(start).A compound function is a sequence of atomic functions over disjoint neighbouring intervals that deﬁne how the evaluation of an intention changes over time.

For example, Bike Lane Usage (see Figure 2.5) can be represented with the compound function illustrated in Figure 3.2, and speciﬁed by the tuple:

{hStochastic,(⊥, ⊥), t0, tspringi, hIncrease,(F, ⊥), tspring, tsummeri,

hConstant,(F, ⊥), tsummer, tfalli, hDecrease,(⊥, F), tfall, tendi}. This consists of four atomic functions, each deﬁning one of the four Canadian seasons (starting with winter at t0): a Stochastic function over [t0, tspring); an Increase function with maximum value

(F, ⊥) over [tspring, tsummer); a Constant function at (F, ⊥) over [tsummer, tfall); and a Decrease function at (⊥, F) over [tfall, tend). The intuition behind choosing these functions is that usage is dependent on the season. Depending on the mildness of winter residents will use bike lanes to varying degrees (Stochastic). There is a steady increase in usage throughout spring (Increase) and a steady decrease in usage throughout fall (Decrease), with consistently high (or satisﬁed) usage in summer (Constant). All time points used to specify the Bike Lanes example are presented together in Table 3.1 for reference.

Definition 3.5 (Repeating Compound Function). Let a sequence of atomic functions A be given, where ai ∈ A is the tuple htype, x, start, stopi and ∀ai ∈ A : ai(stop) = ai+1(start).A repeating compound function is the tuple hRepeat, repNum, absT ime, Ai, where Repeat indicates a repeating function, A is the given set of atomic functions, repNum ∈ N+ is the number of repeats for the entirety of A, and absT ime ∈ N0 is the absolute time for each function or zero if no value is specified. A repeating compound function is a sequence of atomic functions over disjoint neighbouring intervals that repeat repNum to define how the evaluation of an intention g changes over time.

For example, the compound function in Figure 3.2 is repeated three times in Figure 3.3. The last time point in the sequence A is paired with the first time point for the next repeat (i.e., r1(tend) = r2(t0)). We model the first three years of Bike Lane Usage with this repeating function, where we model each new winter (until tspring) with a Stochastic function. The repeating version of Bike Lane Usage is specified by the tuple: hRepeat, 3, 0,

{hStochastic,(⊥, ⊥), t0, tspringi, hIncrease,(F, ⊥), tspring, tsummeri,

hConstant,(F, ⊥), tsummer, tfalli, hDecrease,(⊥, F), tfall, tendi}i. Chapter 3. Modeling Evolving Intentions 39

(F, ⊥)

(⊥, F) r t r r r r r r r r r r r r r r t end 0 1 1 1 2 3 1 1 2 2 2 2 3 3 3 3 ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( t t t t t t t t t t t t t t t fall 0 spring summer fall end 0 summer fall end 0 summer spring spring end ) ) ) ) ) ) ) ) ) ) ) ) ) ) )

Figure 3.3: Sequence of atomic functions in Figure 3.2 repeated three times.

Definition 3.6 (User-Defined Function). Let an intention g ∈ G be given. A User-Defined function is an arbitrary composition of compound and repeating compound functions (Definition 3.4 and 3.5) over disjoint neighbouring intervals (one function per interval) that define how the evaluation of an intention g changes over time.

To most accurately model Bike Lane Usage with a User-Defined function, we should include a period of no usage at the beginning of the function until the bike lanes are built (full evidence against the satisfaction of Bike Lane Usage). The ﬁnal version of the function for Bike Lane Usage is speciﬁed by the tuple:

{hConstant,(⊥, F), t0, tusei, hRepeat, 3, 0,

{hStochastic,(⊥, ⊥), t0, tspringi, hIncrease,(F, ⊥), tspring, tsummeri,

hConstant,(F, ⊥), tsummer, tfalli, hDecrease,(⊥, F), tfall, tendi}i}, where tuse is deﬁned as when the construction of the bike lanes is ﬁnished.

Common Compound Functions. Given the User-Defined function, we can define for convenience the functions listed in Table 3.2, which we call common compound functions. For example, in Table 3.2, the Constant-Stochastic function definition consists of two atomic functions: first, a

Constant function over the interval [e0, e1) with a constant value of x, and second, a Stochastic function over the interval [e1, e2). (⊥, ⊥) is used as the reference evidence pair in the interval [e1, e2), because Stochastic does not require a value. For the Bike Lanes example (see Figure 2.5), we deﬁne Have Design as having a Denied-Satisfied function: hDenied-Satisfied,(F, ⊥), {0, tdesign, maxT ime}i, which is the same as the User-Defined function: {hConstant,(⊥, F), t0, tdesigni, hConstant,(F, ⊥), tdesign, tendi}, where tdesign is the time when Have Design becomes satisﬁed and a design has been created. So the value of Have Design is (⊥, F) until a design is accepted at tdesign, and then it is (F, ⊥). The intuition behind this function is that a tasks starts not completed and remains so until its completion time. Chapter 3. Modeling Evolving Intentions 40

Table 3.2: List of common compound functions. Each function is the tuple htype, x, Ei, where type is one of the types listed in the leftmost column, x ∈ E is a reference evidence pair that may be used in the function, and E ⊆ P is a sequence of three time points. The rightmost column speciﬁes how the parameters are mapped to the User-Defined function as two atomic functions. In this column, atomic function types are identiﬁed by short forms I:Increase, D:Decrease, C:Constant, and R:Stochastic.

Type Usages Deﬁnition

Satisfied- the evidence pair assignment remains (F, ⊥) and {hC,(F, ⊥), e0, e1i, Denied then remains (⊥, F) hC,(⊥, F), e1, e2i}

Denied- the evidence pair assignment remains (⊥, F) and {hC,(⊥, F), e0, e1i, Satisfied then remains (F, ⊥) hC,(F, ⊥), e1, e2i}

Stochastic- the evidence pair assignment is stochastic or random {hR,(⊥, ⊥), e0, e1i, Constant and then remains constant at x hC, x, e1, e2i}

Constant- the evidence pair assignment remains constant at x {hC, x, e0, e1i, Stochastic and then changes are stochastic or random hR,(⊥, ⊥), e1, e2i}

Monotonic changes in evidence pair assignment become “more {hI, x, e0, e1i, Positive true” to x and then remains constant at x hC, x, e1, e2i}

Monotonic changes in evidence pair assignment become “less {hD, x, e0, e1i, Negative true” to x and then remains constant at x hC, x, e1, e2i}

3.1.3 Evolving Functions for Relationships

In addition to having evolving intentions, we allow stakeholders to specify evolving relationships.

Deﬁnition 3.7 (N-ary Evolving Relationship). Let intentions (g1, ..., gn), g ∈ G, relationships r1, r2 ∈ {and, or, no}, a set of time points P , and a reference time point tRef ∈ P be given. An n-ary r1/r2 relationship (g1, ..., gn) −−−→ g is evolving with respect to tRef if tRef

 r1 (g1, ..., gn) −−−→ g, if ti < tRef ∀ti ∈ P r2 (g1, ..., gn) −−−→ g, otherwise.

This allows stakeholders to change the nature of a decomposition relationship. For example, assume initially that bike lanes can be identified by either painted lines or flexible bollards, but a change in government regulations requires bike lanes to be identified by both painted lines and flexible bollards. or/and This relationship between bike lanes and its identifying components can be represented with a −−−→ ti link, where over the same intentions the or relationship holds until some ti and then the and relationship holds.

Deﬁnition 3.8 (Binary Evolving Relationship). Let intentions g1, g ∈ G, relationships r1, r2 ∈ {++, +, --, -,

++S, +S, --S, -S, ++D, +D, --D, -D, no}, a set of time points P , and a reference time point tRef ∈ P be r1/r2 given. A binary relationship g1 −−−→ g is evolving with respect to tRef if tRef

 r1 g1 −−−→ g, if ti < tRef ∀ti ∈ P r2 g1 −−−→ g, otherwise. Chapter 3. Modeling Evolving Intentions 41

Similarly, the strength or type of a contribution relationship may change over time. This evolution changes the impact one goal may have on another. For example, choosing a design with Permanent Con- struction will impact Access to Parking but only while the construction is taking place. After construction is complete and the bike lanes become open (at tBLOpen), Permanent Construction has no impact on Access to Parking. We can model the relationship between Permanent Construction and Access to Parking with an evolving --S/no relationship. This relationship, denoted by --S/no Permanent Construction −−−−→ Access to Parking, tBLOpen allows us to model the negative impact of construction on Access to Parking, where signiﬁcant road work is required until tBLOpen, which denotes the time when the construction is complete and the bike lanes become open. Prior to time point tBLOpen if Permanent Construction is fully (or partially) satisﬁed, then there exists full (or partial) evidence that Access to Parking is denied, and after tBLOpen no evidence is propagated.

NOTBOTH Function. We have created a new relationship function speciﬁcally for the purpose of showing when a mutually exclusive decision is made in the graph over time. This relationship guarantees that only one alternative will be chosen in any analysis procedure, where prior to making the decision, each alternative did not impact other intentions. We have two variations that specify the evidence pair for the unchosen alternative: NotBoth-Denied (NBD) where the unchosen alternative is assigned (⊥, F), and NotBoth-None (NBN) where the unchosen alternative is assigned (⊥, ⊥).

Definition 3.9 (NotBoth Evolving Function). Let intentions g1, g2 ∈ G, a set of time points P , and a reference time point tRef ∈ P be given, where g1 and g2 are not elements of any atomic functions. x is a special evidence pair used to define the final value, where x = (⊥, F) for NBD and x = (⊥, ⊥) for

NBN. If there exists a relationship between intentions g1 and g2 such that  ( g [t ] = (⊥, ⊥) ∧ g [t ] = (⊥, ⊥)), if t < t  1 i 2 i i Ref  J K J K    (( g1[ti] = (F, ⊥) ∧ g2[ti] = x ∨ ∀ti ∈ P , J K J K ( g [t ] = x ∧ g [t ] = (F, ⊥)), if t = t  1 i 2 i i Ref  J K J K    ( g1[ti] = g1[tRef] ∧ g2[ti] = g2[tRef ), otherwise. J K J K J K J K NBD NBN We denote NotBoth Evolving Function as g1 ←−−→ g2 and g1 ←−−→ g2. tRef tRef

For example, we can represent the mutually exclusive decision between Permanent Construction and Temporary Construction by modifying the graph in Figure 2.5 with the NotBoth evolving function NBN Permanent Construction ←−−−→ Temporary Construction, where tdecision denotes the time when a decision tdecision between Permanent Construction and Temporary Construction is made. Prior to the time point tdecision, no decision has been made, and both intentions have the evidence pair (⊥, ⊥). After tdecision, only one intention will have the evidence pair (F, ⊥), and the other will have (⊥, ⊥). This relationship means that only one of the alternatives Permanent Construction and Temporary Construction can be chosen. The chosen alternative becomes satisﬁed and can impact other intentions, while the other alternative will not impact other intentions. Chapter 3. Modeling Evolving Intentions 42

3.1.4 Specifying Evolving Models

Stakeholders can specify constraints between time points and give absolute time assignments to time points.

Deﬁnition 3.10 (Graph Constraints). Let time points t1, t2 ∈ P be given. A graph constraint is a relationship between t1 and t2, such that the graph constraint is of the form t1 < t2, t1 = t2, t1 = k, or 0 t1 6= k, where k ∈ N ∧ k ∈ [0, maxT ime).

In Table 3.1, we described tdesign, tdecision, tBLOpen, and tuse, which are all used in the Bike Lanes example. We can deﬁne constraints between these times, such that tdesign < tdecision, tuse = tBLOpen =

10, and tdecision = 5. Graph constraints do not need to be associated with evolving elements in the graph. Users can name time points and assign absolute values to them. For example, in Table 3.1, we added the constraints ta0 = 7, ta1 = 14, and ta2 = 34. in Chapter4. Graph constraints cannot be added inside any repeating segment of a User-Defined function because these repeating time points are not explicitly part of the goal graph speciﬁcation. We return to this point in Chapter4.

We extend the deﬁnition of a goal graph (see Deﬁnition 2.6) to include evolutionary concepts.

Definition 3.11 (Evolving Goal Graph). An evolving goal graph is a tuple hA, G, R, EF , MC, maxTimei, where A is a set of actors, G is a set of intentions, R is a set of relationships over intentions (including evolving relationships and NotBoth evolving functions), EF is a set of mappings between elements in G and User-Defined functions (see Definition 3.6), MC is a set of graph constraints over the evolving functions and relationships, and maxTime ∈ N+ is the maximum absolute time over which any evolving function is defined. A well-formed evolving goal graph is a graph where: (1) each intention has at most one incoming n-ary relation, and (2) the graph does not contain any loops. From this point on, when we say “goal graph”, we mean a well-formed evolving goal graph.

With this new definition, we update the goal graph from Figure 2.5 with the evolving elements and present the new goal graph in Figure 3.4, with the specification in Figure 3.5. Each element in the visual graph maps to a portion of the specification. The intentions and relationships are listed first. Notice the specification of the relationships R in Figure 3.5 now has the evolving relationship and the NotBoth evolving functions (discussed in Section 3.1.3 and 3.1.3, respectively). This graph has only two evolving functions (see EF in Figure 3.5), one for Have Design and a much longer one for Bike Lane Usage (both explained in detail in Section 3.1.2). The last line lists the constraints added between time points in the graph. Chapter 3. Modeling Evolving Intentions 43

Legend DS Denied-Satisfied UD User-Defined NB NotBoth Relationship Intentions

Figure 3.4: Slice of Bike Lanes example goal graph with evolving elements.

The Bike Lanes Goal Graph is hA, G, R, EF , MC, maxTimei where, A = {}, G = {hHave Bike Lanes, goali, hHave Design, goali, hBuild Bike Lanes, goali, hTemporary Construction, goali, hPermanent Construction, goali, hCyclist Safety, goali, hAccess to Parking, goali, hBike Lane Usage, goali}, and R = {(Have Design, Build Bike Lanes) −−−→ Have Bike Lanes, or (Temporary Construction, Permanent Construction) −−−→ Build Bike Lanes, ++S Have Bike Lanes −−−→ Cyclist Safety, --S/no Permanent Construction −−−−−→ Access to Parking, tBLOpen NBN Permanent Construction ←−−−→ Temporary Construction}, tdecision

EF = {hHave Design, hDenied-Satisfied,(F, ⊥), {t0, tdesign, tend}ii, hBike Lane Usage, {hConstant,(⊥, F), t0, tusei, hRepeat, 3, 0, {hStochastic,(⊥, ⊥), t0, tspringi, hIncrease,(F, ⊥), tspring, tsummeri, hConstant,(F, ⊥), tsummer, tfalli, hDecrease,(⊥, F), tfall, tendi}i}i},

MC = {tdesign < tdecision, tuse = tBLOpen = 10, tdecision = 5, ta0 = 7, ta1 = 14, ta2 = 34}, and maxTime = 50. Figure 3.5: Speciﬁcation of a fragment of Bike Lane example goal graph with evolving elements. Chapter 3. Modeling Evolving Intentions 44

3.1.5 Completeness

In the previous subsections, we describe how to model with the Evolving Intentions framework. But, the set of function types is not complete for describing all possible evolutions. The following list gives examples that cannot be expressed in the framework: • Atomic functions for intentions with presence conditions1. • Repeating functions with an arbitrary number of repeats. • Repeating functions that should repeat until an independent event (e.g., an intention is assigned a specific evidence pair). • Repeating functions with absolute time points specified during the repeating portion, or absolute time periods for individual segments in the repeat. • Evolving relationships with more than two relationships. • Mixing n-ary and binary evolving relationship types. • More than two intentions involved in a single NotBoth relationship. Through the empirical validation in Chapter6 and Chapter7, we show that the atomic and user-defined functions have sufficient expressive power for modeling the examples and cases we investigate. We expect the usability of function types to be improved as new common compound function types are defined through usage. These new common compound functions can be added to existing tooling through a macro generation feature.

1This is partially supported, see Section 7.4.2 for details. Chapter 3. Modeling Evolving Intentions 45

Legend R Stochastic RC Stochastic-Constant DS Denied-Satisfied CR Constant-Stochastic UD User-Defined

Figure 3.6: Model for the Waste Management example (WME) with evolving function labels.

3.2 Modeling in iStar

In this section, we demonstrate how the Evolving Intentions framework can be adapted for iStar with seven qualitative evaluation labels (see Section 2.1.3 for details). We illustrate this process using the model in Figure 1.1 of the Waste Management example (WME), introduced in Section 1.2.1. As we review evolutionary concepts, we reference the final WME model shown in Figure 3.6 (see Figure 3.8 for the complete specification). There is nothing principally different about applying the Evolving Intentions framework to iStar from what we presented for Tropos in Section 3.1. The same intuition carries for Definition 3.1 through Defini- tion 3.11. iStar does not have formal semantics, so we do not repeat formal definitions here, but instead discuss how an evolving goal graph, with evolving functions for intentions and relationships, would be specified in iStar. As introduced in Chapter2, there are two key differences between Tropos and iStar. First, iStar uses different contribution links (e.g., makes) and has dependency links. This change impacts the r1/r2 definition of Binary Evolving Relationship (see Definition 3.8) specified as g1 −−−→ g, where in iStar tRef r1, r2 ∈ {makes, helps, hurts, breaks, depends, no}. Second, as mentioned above, iStar uses seven qualitative evaluation labels (i.e., Fully Satisfied (FS), Partially Satisfied (PS), Partially Denied (PD), Fully Denied (FD), Conflict (CF ), Unknown (U ), and None (N )) resulting in different semantics for evaluation at a time point and all functions for intentions (see Definition 3.1–3.6), as well as the definition of NotBoth (see Definition 3.9) and an evolving goal model (see Definition 3.11), but the intuition for these definitions remains. As a result, there is no notion of a conflict prevention level, but instead, Conflict and Unknown are only assigned via propagation (see Chapter4 for details). Chapter 3. Modeling Evolving Intentions 46

In GrowingLeaf, the implementation of the Evolving Intentions framework for iStar, there is no functionality for the NotBoth evolving function, and there is no concept of absolute time (i.e., all time points are relative). In the remainder of this section, we use placeholders for absolute times in repeating compound functions (see Deﬁnition 3.5) and maxTime in Deﬁnition 3.11, but these values will not be used. Using Evolving Functions for Intentions. Here we describe the evolving functions that are assigned to intentions in Figure 3.6. We begin by looking at Environmental Concern. We are certain that the Citizens’ concerns about the environment will change over time, but we cannot yet predict how these views will change; thus, we assign Environmental Concern a Stochastic function (denoted by R in Figure 3.6). We assign several common compound functions, for example Build Green Centre is assigned a Denied-

Satisfied function, meaning Build Green Centre is Constant with the value FD from t0 until tBGC and then Constant with the value FS until tend. Build Green Centre is treated as a task that is completed at a single time point. When the future evaluation is known but the intermediate evaluations are not known, we model intentions with a Stochastic-Constant function, where the value is unknown (Stochastic) until a time point and then it is known with a user-specified constant value (Constant). In Figure 3.6, both Update Truck Route and Use New Dump are assigned Stochastic-Constant functions. In this case, the City believes that the truck route will eventually be updated allowing the trucks to use the new dump, but does not make predictions about the evaluation prior the update. On the flip side, the City may only know the evolution of an intention for a period of time. In this case, a Constant-Stochastic function defines that the evaluation of an intention is a user-specified constant value (Constant) until a time point and then a Stochastic function. Comply with Standards is assigned a Constant-Stochastic function because until the new standard is released, the City does not need to comply with anything and has no information so it has the evaluation None (N ).

Figure 3.7: An illustration of the User-Defined function type for GW Education Program.

We also use User-Defined functions to specify intentions. GW Education Program is assigned a User-Defined function (illustrated in Figure 3.7), where the evaluation is Constant with the value

FD from t0 until tBGC, Constant with the value PS until tSRP, Constant with the value FS until tERP, and Constant with the value PS until tend. This illustrates that the education program will not exist until the green centre is built, and then the program will ramp up, becoming partially satisfied and then fully satisfied, and then ramp down becoming partially satisfied again. Finally, Have Workers Union Contract is assigned a repeating User-Defined function following the pattern where the contract is available (Constant with a FS value) for an interval, then not available (Constant with a FD value) for another interval, then again available and so on. Each of these functions is specified in the EF set in Figure 3.8. Using Constraints and Evolving Functions for Relationships. We use evolving functions and relationships to represent model evolution where goals are removed from the model and/or new goals Chapter 3. Modeling Evolving Intentions 47

appear in the model. For example, if we want the goal Comply with Standards to appear at tCWS when the new standards are released, we add the element to the original model and give it a function type (such as Constant-Stochastic described above) where the initial interval is a Constant function with the value N until tCWS. We also add that Comply with Standards to Manage City Waste has no relationship until tCWS and a Dependency relationship after. Thus, we use the N evaluation and the no relationship to represent intentions not present over specific intervals. We can also add constraints between time points. For example, we know that Update Truck Route is completed (FS) before either the Use New Dump or the Process Green Waste tasks are FS, but neither depends on an updated route. Given that Update Truck Route is modeled as Stochastic-Constant with the value FS at tUTR, and Use New Dump is modeled as Stochastic-Constant with the value FS at tUND, we can add the constraint that tUTR occurs before tUND (i.e., tUTR < tUND). Waste Management example Model with Evolution. Consider the definition of an Evolving Goal Graph (see Definition 3.11), where an evolving goal model is a tuple hA, G, R, EF , MC, maxTimei. If we adapt this definition for an iStar Goal Model (see Definition 2.1), where R contains iStar links (i.e., and, or, makes, helps, hurts, breaks, depends) and EF uses the iStar evaluation labels (see Defini- tion 2.1.3), then we can specify the evolutionary information in an iStar model. Figure 3.8 gives the full specification of the Waste Management example with the details discussed in this section (see Figure 3.6 for the visual representation).

3.3 Summary

In this chapter, we presented a formal approach to modeling with evolution, and ultimately gave a definition for an Evolving Goal Graph in Tropos. We first established how time is represented and then defined general functions for specifying how intentions and relationships change over time, as well as constraints between time points. For each of these, we introduced a notation, specified its usage and how each function is represented visually, and finally described each component of the framework using the Bike Lanes example. We then compared how an iStar implementation of the Evolving Intentions framework differs from the framework definitions in Tropos. Finally, we described how stakeholders might use the Evolving Intentions framework to specify changes in iStar using the Waste Management example. Chapter 3. Modeling Evolving Intentions 48

The speciﬁcation of the WME model in Figure 3.6 based on the descriptions provided in this section is hA, G, R, EF , MC, maxTimei where, A = {hCity, actor, {Manage City Waste, Use Current Dump, Use New Dump, Build Small Dump, Build Large Dump, Process Green Waste, Reduce Operating Costs, Space in Dump, Build Green Centre, GW Education Program, Have Workers Union Contract, Update Truck Route, Update Truck, Comply with Standards, Purchase Land, Positive City Image, Separate Centre Inputs}i, hCitizens, actor, {Environmental Concern, Quality of Waste Separation, Willingness to Separation Waste, Be a Model Citizen, Separate Waste, Learn Rules for Separation}i}, G = {hManage City Waste, goali, hUse Current Dump, taski, hUse New Dump, taski, hBuild Small Dump, taski, hBuild Large Dump, taski, hProcess Green Waste, goali, hReduce Operating Costs, soft-goali, hSpace in Dump, resourcei, hBuild Green Centre, taski, hGW Education Program, taski, hHave Workers Union Contract, goali, hUpdate Truck Route, taski, hUpdate Truck, taski, hComply with Standards, goali, hPurchase Land, resourcei, hPositive City Image, soft-goali, hSeparate Centre Inputs, taski, hQuality Waste Separation, soft-goali, hSeparate Recycling, taski, hComplete Waste Separation Training, taski, hEnvironmental Concern, soft-goali, hQuality of Waste Separation, soft-goali, hWillingness to Separation Waste, soft-goali, hSeparate Waste, taski, hBe a Model Citizen, goali, hLearn Rules for Separation, taski}, or R = {(Use Current Dump, Use New Dump) −−−→ Manage City Waste, or (Build Small Dump, Build Large Dump) −−−→ Use New Dump, and (Build Green Centre, Upgrade Trucks, Separate Centre Inputs) −−−→ Process Green Waste, and helps (Space in Dump) −−−→ Use Current Dump, Process Green Waste −−−→ Positive City Image, helps hurts Build Small Dump −−−→ Reduce Operating Costs, Build Large Dump −−−→ Reduce Operating Costs, hurts Process Green Waste −−−→ Reduce Operating Costs, and (Separate Waste, Learn Rules for Separation) −−−→ Be a Model Citizen, helps Learn Rules for Separation −−−→ Willingness to Separation Waste, makes Be a Model Citizen −−−→ Willingness to Separation Waste, makes Willingness to Separation Waste −−−→ Quality of Waste Separation, makes Environmental Concern −−−→ Quality of Waste Separation, depends Process Green Waste −−−→ Quality Waste Separation, depends Quality Waste Separation −−−→ Quality of Waste Separation, depends depends Separate Centre Inputs −−−→ Separate Recycling, Separate Recycling −−−→ Separate Waste, depends Learn Rules for Separation −−−→ Complete Waste Separation Training, depends Complete Waste Separation Training −−−→ GW Education Program, depends/no Manage City Waste −−−−−−→ Comply with Standards}, tMCR

EF = {hBuild Green Centre, hDenied-Satisfied, FS, {t0, tBGC , tend}ii, hUse New Dump, hStochastic-Constant, FS, {t0, tUND, tend}ii, hUse Current Dump, hConstant-Stochastic, FS, {t0, tUCD, tend}ii, hUpdate Truck Route, hStochastic-Constant, FS, {t0, tUTR, tend}ii, hComply with Standards, hConstant-Stochastic, N, {t0, tCWS, tend}ii, hEnvironmental Concern, hStochastic, N, t0, tendii, hGW Education Program, {hConstant, FD, t0, tBGC i, hConstant, PS, tBGC , tSRP i, hConstant, FS, tSRP , tERP i, hConstant, PS, tERP , tendi}i, hHave Workers Union Contract, hRepeat, 2, 0, {hConstant, FS, t0, tEoC i, hConstant, FD, tEoC , tendi}ii},

MC = {tCWS = tMCR, tUTR < tUND}, and maxTime = 25.

Figure 3.8: Speciﬁcation of the Waste Management example (WME) with evolving elements. Chapter 4

Reasoning with Evolving Intentions

Recall that the goal of this dissertation is to allow the evaluation of intentions to change over time and enable stakeholders to answer time-based questions by creating simulation paths. In Chapter3, we introduced the modeling portion of the Evolving Intentions framework. We described how the evaluation of an intention can change over time and defined an evolving goal graph. In this chapter, we combine the analysis from Chapter2 (e.g., propagation of evaluations throughout intentions in a goal graph) with the evolving functions described in Chapter3 to define how goal graphs are evaluated at specific time points (states) and how these evaluations are combined to create a simulation path (i.e., evaluations of a graph across a series of time points). We present this reasoning in Tropos, and illustrate these definitions with the Bike Lanes example, and specify an algorithm for finding Graph Evaluation Paths, which can be used to answer modelers questions. We then present an additional algorithm for finding paths when evolving intentions are only applied to leaf-level intentions in iStar. While this second algorithm is not as robust, it has a faster runtime and can be used for some stakeholder questions, as well as a general exploration of the goal graph. To connect simulation generation with questions asked by stakeholders, we introduce four simulation strategies and demonstrate how they can be used to answer the modeler questions posed in Section 1.2 for the Bike Lanes example and Waste Management example.

Contributions

This chapter makes the following contributions. • We present a formal approach to reasoning with evolution in Tropos using the Bike Lanes example. • We define the meaning of a time point path and how a goal graph can be evaluated at each time point, giving a definition for a Graph Evaluation Path. • We provide a brute force algorithm in Tropos for finding graph evaluation paths and prove properties of the algorithm. • We discuss a constant time algorithm in iStar for finding graph evaluation paths when evolution is restricted to leaf-level intentions. • We introduce four simulation strategies and describe how stakeholders can use the Evolving In- tentions framework to answer time-based questions using the Bike Lanes example and the Waste Management example, on Tropos and iStar, respectively.

49 Chapter 4. Reasoning with Evolving Intentions 50

Organization

The remainder of this chapter is organized as follows: Section 4.1 deﬁnes the notion of an evolving goal graph path and an algorithm for ﬁnding graph paths in Tropos. Section 4.2 discusses how to compute a goal graph path with only evolving leaf-level intentions in iStar. Section 4.3 introduces our simulation strategies to answer stakeholder questions. Section 4.4 provides a summary of the results.

4.1 Reasoning in Tropos

We ended Section 3.1 with the definition of an Evolving Goal Graph (see Definition 3.11). In this section, we define how Evolving Goal Graphs can be evaluated over time creating Graph Evaluation Paths.

4.1.1 Graph Evaluation

Designating Graph Time Points. To create a simulation of a goal graph, we first establish which time points should exist in the path (i.e., simulation). Time points are derived from the symbolic and absolute constants defined in model constraints as well as functions for evolving intentions and relationships (i.e., MC, EF, R). From the specification of the Bike Lanes example (BLE) in Figure 3.5, we list all relevant constraints for time points in Table 4.1, as well as the origin of each constraint. For example, the first table row lists the constraints associated with the evolving function for Have Design, where t0 < tdesign < tend. The last table row in Table 4.1 lists all the constraints from the MC specification in Figure 3.5. The second row in Table 4.1 lists the partial order over the time points associated with the evolving function for Bike Lane Usage. In its specification (see Figure 3.5), Bike Lane Usage contains four atomic functions that repeat three times. Prior to performing any analysis on the goal graph, we first unroll repeating segments and create new time points for each repeat of the form ri(tk) for each repeat i and time point k. In the example, the first repeat starts with r1(tuse) and ends with r1(tend). The end of each repeat is set to be equal to the start of the next repeat, so r1(tend) = r2(tuse) and so on. Notice that tspring, tsummer, and tfall do not appear in Table 4.1. Since these time points are internal to the repeat, they are removed, and are not allowed in constraints between time points (see Section 3.1.4).

Thus, using tspring or any repeating time point (e.g., r2(tspring)) in a graph constraint is not allowed. With the constraints in Table 4.1, we can create a partial order over all the time points in the graph.

Deﬁnition 4.1 (Goal Graph Time Points). Let a goal graph M = hG, R, EF , MC, maxTimei be given. Let TP (M) be the set of all time points named in the speciﬁcation of M, called the graph time points. TP (M) returns the time points with unrolled repeating segments.

Given the specification of the Bike Lanes example (see Figure 3.5), TP (M) would return the time points listed in Table 4.1 (and Table 4.2). User Evaluations. We define what it means to evaluate an intention at a time point by extending Definition 2.10 and 2.11. Evidence pairs can be assigned to intentions through propagation or user assignments. Stakeholders can assign an evidence pair to any intention in the graph at any time point Chapter 4. Reasoning with Evolving Intentions 51

Table 4.1: Constraints associated with the Bike Lane example. The table lists all the constraints between time points and the origin of the constraints, for the Bike Lanes example. In this example, maxT ime is assigned arbitrarily to be 50, so tend = 51 (and, as always, t0 = 0).

Graph Constraints Between Time Points Origin

t0 < tdesign < tend Denied-Satisfied function: Have Design

t0 < (tuse = r1(tuse)) < r1(tspring) < r1(tsummer) < User-Defined function: Bike Lane r1(tfall) < (r1(tend) = r2(tuse)) < r2(tspring) < Usage r2(tsummer) < r2(tfall) < (r2(tend) = r3(tuse)) < r3(tspring) < r3(tsummer) < r3(tfall) < (r3(tend) = tend)

t0 < tBLOpen < tend --S/NO relationship between Perma- nent Construction & Access to Parking

t0 < tdecision < tend NBT relationship between Permanent Construction & Temporary Construction

tdesign < tdecision Graph Constraints

tuse = tBLOpen = 10

tdecision = 5

ta0 = 7

ta1 = 14

ta2 = 34 in the Graph Time Points (see Deﬁnition 4.1). Stakeholders assign known evidence pairs at the current time point (t0). Additionally, stakeholders can assign desired values of intentions at other time points to constrain simulation results. However, by doing so, stakeholders run the risk of over-constraining the analysis resulting in no valid path.

Deﬁnition 4.2 (User Evaluation at a Time Point). Let a goal graph M = hG, R, EF , MC, maxTimei be given. A set of user evaluations UEval = {hg, t, ei | g ∈ G ∧ t ∈ TP (M) ∧ e ∈ E} is the user’s assignment of evidence pairs to intentions at time points.

In the Bike Lanes example, a set of user evaluations is {hHave Design, ta2 ,(F, ⊥)i, hCyclist Safety, t0,(⊥, P)i}. User evaluations can be any combination of intention and time point. For example, we could include all intentions at t0 to assign the current values that the user knows to be true in the real world. The user can also assign desired values (e.g., root-level goals to achieve in the future), such as saying Cyclist Safety will be satisfied at ta1 , as in hCyclist Safety, ta1 ,(F, ⊥)i. However, this same value can be assigned with an evolving function. For example, assigning Cyclist Safety a Stochastic-Constant that is constant with the evidence pair (F, ⊥) at ta1 would achieve the same result. It is possible for users to add user affirmed intentions that make the evaluation of the goal graph inconsistent. For example, if we added the user evaluation hCyclist Safety, ta2 ,(⊥, P)i in conjunction with the Stochastic-Constant function, this would result in an inconsistency in the value of Cyclist Safety at ta2 . Evaluating An Evolving Goal Graph at a Time Point. With the introduction of User Evaluations and Graph Time Points, we can now evaluate a goal graph at a time point. In Section 2.2.5, we defined a goal graph evaluation (see Definition 2.11). We extend this definition to add timing information that allows goal graphs to exist at a time point, which defines our notion of a state. A set of goal graph Chapter 4. Reasoning with Evolving Intentions 52 evaluations at time points are connected to create a path that describes the evolution of a goal graph in Section 4.1.2.

Deﬁnition 4.3 (Goal Graph Evaluation at a Time Point). Let a well-formed goal graph M = hG, R, EF ,

MC, maxTimei, a set of user evaluations UEval, a reference time point tRef ∈ TP (M), and a conﬂict level CFLevel be given. A complete evaluation of M at tRef is the total mapping G × tRef → E resulting from the repeated application of Deﬁnition 3.2–3.10 to all intentions, assignment of evidence pairs to intentions, and application of the propagation rules in Figure 2.8 to all relationships in R, given UEval such that ∀g ∈ G : g[tRef ] 6∈ CFLevel ∧ g[tRef ] 6= ⊥ ∧ ( ∀hg, tRef , ei ∈ UEval : e = g[tRef ] ). When J K J K J K this is not possible, the evaluation of the goal graph is said to be infeasible.

We show the evaluation of the Bike Lanes example at tdecision in Figure 4.1 and ta1 in Figure 4.2. These time points were selected to showcase both a time point where there is a transition in an evolving function (i.e., tdecision is defined in Have Design) and an absolute time point (i.e., ta1 is specified in MC). For both examples, we did not have user evaluations, and the CFLevel was set to Weak Conflict.

In Figure 4.1, tdecision = 5, Have Design’s Denied-Satisfied function has already transitioned to (F, ⊥) because tdesign < tdecision, and Bike Lane Usage is (⊥, F) because tdecision < tuse. At tdecision, a decision is made between Permanent Construction and Temporary Construction. In this evaluation, Permanent Construction is chosen and assigned (F, ⊥), and Temporary Construction remains (⊥, ⊥). The values for the rest of the intentions in the graph are then propagated from these evidence pairs. Access to Parking has the value (⊥, F) because of the --S relationship with Permanent Construction, which has not evolved yet because tdecision < tBLOpen. Figure 4.2 shows an evaluation of the goal graph at time ta1 = 14 (which is after tBLOpen). At this point, Bike Lane Usage is (P, ⊥) and in one of the repeating portions of its function, and the evolving relationship between Permanent Construction and Access to Parking has transitioned to NO, so the value of Access to Parking is not propagated from Permanent Construction. The value of Access to Parking is non-deterministic and in this evaluation has the value (⊥, ⊥). The remainder of the intentions have the same evidence pairs as in Figure 4.1 at tdecision. From these two time points, we see how the evaluation of the goal graph changes between tdecision and ta1 .

Both of the evaluations at tdecision and ta1 are complete (see Deﬁnition 4.3). Suppose instead we added the user evaluation hHave Design, ta1 ,(⊥, P)i. This would result in an infeasible graph evaluation at ta1 because the value of Have Design is over-constrained. The Denied-Satisfied function deﬁnes the value of Have Design as (F, ⊥) after tdesign, and ta1 is after tdesign, so the user cannot assign the value of Have Design to be (⊥, P). Infeasible graph evaluations help the user understand which evaluations are not possible and debug the graph when errors are made.

4.1.2 Generating Simulation Paths

In this section, we discuss path generation and evaluation of a goal graph along a path. A state is an evaluation of a goal graph at a given time t.A path is a sequence of states where time is increasing and all time points with the same time have identical evaluations of the goal graph. A path is an ordering of existing constraints over the time points in the graph. A path is only generated once all repeating segments of User-Defined functions have been unrolled (see Deﬁnition 4.1), because this results in the creation of additional time points. Chapter 4. Reasoning with Evolving Intentions 53

Figure 4.1: Evaluation of the Bike Lane example Figure 4.2: Evaluation of the Bike Lane example at tdecision = 5. after ta1 = 14.

Definition 4.4 (Time Point Path). Let a goal graph M, and a set of goal graph time points in M (i.e., TP (M)) be given. A time point path Π is a partial order over TP (M) satisfying the graph constraints in M, where Π = {ht, t i | t ∈ TP (M) ∧ t 6= ⊥ ∧ t ≤ maxTime}. J K J K J K A time point path is created by assigning an absolute value to each time point in the goal graph TP (M). For example, we can assign an absolute time to each of the time points listed in Table 4.1 for the Bike Lanes example. A time point path is shown in the first two columns in Table 4.2. Other time point paths can exist by assigning different absolute values to each time point.

Recall that tend = maxTime + 1 is included in TP (M) with the rationale that we deﬁned intervals over time points that are open on the right, and we want to make sure that maxT ime can be included

(see Section 3.1.1). However, the time point path explicitly does not include tend because it is greater than maxTime. With a time point path we can evaluate the graph over each of the time points in the path to create a simulation of the evolution of the graph.

Deﬁnition 4.5 (Graph Evaluation Path). Let a goal graph M, a time point path Π, a set of user evaluations UEval, and a conﬂict level CFLevel be given. A complete graph evaluation path is the complete evaluation of M at each time point in Π, given UEval and CFLevel.

Formally, ∀t ∈ Π: G[t] → E such that ∀g ∈ G : g[t] 6∈ CFLevel ∧ g[t] 6= ⊥ and the rules in Figure 2.8 J K J K and Deﬁnition 3.2–3.10 are respected.

Once computed, the graph evaluation path consists of two parts: first, the time point path Π with each time point having an absolute value assigned to it; and second, the set of intentions in the graph have an assigned evidence pair for each time in the time point path (i.e., for each time point in Π). In the remainder of this dissertation and in BloomingLeaf, we refer to this analysis process for finding a Graph Evaluation Path that satisfies Definition 4.5 as Simulation over All Evolving Intentions.

Partial Path. We can construct a partial time point path for a graph by choosing a subset of the time point path, and evaluating the graph at each time point in the partial path. Chapter 4. Reasoning with Evolving Intentions 54

4.1.3 Graph Evaluation Path Algorithms

In this section, we present Algorithm 4.3, which ﬁnds a complete Graph Evaluation Path (satisfying Def- inition 4.5) if one exists, and two helper functions (i.e., Check EI Func and Check NotBoth) used in the algorithm to check the evolving functions in the evolving goal graph.

CHECK EI FUNC Function (Algorithm 4.1). We begin with Check EI Func (see Algorithm 4.1) where the purpose is to check that no evolving functions for intentions have been violated while assigning evidence pairs. This boolean function takes as input a set of goal assignments aGoals, the time point path Π, and EF which consists of mappings between intentions g and User-Defined function (see Def- inition 3.6) consisting of atomic functions (see Definition 3.2), and returns true if no atomic functions have been violated. The main loop (see Lines2–20) iterates over each item in the set EF , which is of the form hg, {htype0, x0, start0, stop0i, ..., htypen, xn, startn, stopni}i. The function loop (see Lines4–20) then iterates over each function (initializing values on Line5) and checks each value of the time point path Π within the range [start, stop) depending on the type (i.e., Constant, Increase, Decrease). For example, the check for Constant (see Lines6–8) verifies that each assigned evidence pair (i.e., having a value in aGoals) is equal to x. If all checks pass and the main loop completes, Check EI Func returns true on Line 21, but if a single check fails, false is returned. As mentioned when discussing Definition 2.11 (see Section 4.1.1) if stakeholders add the user evaluation in conjunction with an evolving function, this results in an inconsistency. These inconsistencies are found by Check EI Func. For example, if Have Bike Lanes has the evolving function

hConstant,(F, ⊥), tBLOpen, tendi, and is accidentally assigned the user evaluation hHave Bike Lanes, ta2 ,(⊥, F)i, where tBLOpen is before ta2 , then this inconsistency would be found by Check EI Func on Line8 returning false. As discussed in Section 2.2.4, we calculated runtimes in terms of the number of intentions in a goal graph (i.e., n = |G|). Let m be the number of time points in the Time Point Path Π (see Deﬁnition 4.4). When Check EI Func is called, it checks every time point for each atomic function in EF . In the worst case, every intention has a new atomic function for every time point; thus, the worst case runtime of Check EI Func is O(n ∗ m).

CHECK NOTBOTH Function (Algorithm 4.2). Similarly, Check NotBoth inspects the assignments for both intentions in each NotBoth relationship function in the evolving goal graph (see Al- gorithm 4.2). Check NotBoth returns true iff expected values of all assignments are consistent with Definition 3.9. NotBoth relationships are stored in the set of relationships R; thus, the main loop (see Lines2–20) iterates over each relationship and only investigates further for NBN and NBD relationships. After initializing relevant values on Lines3–6, the time point loop (see Lines7–20) iterates over each time point in Π. For time points before the decision (t < tRef ), Check NotBoth verifies that assigned intentions (i.e., intentions having values in aGoals) are (⊥, ⊥) (see Lines8–11). For the decision time point tRef , Check NotBoth verifies that one intention is (F, ⊥) and the other has fVal, the value specified by the type, where fVal is (⊥, ⊥) for NBN and (⊥, F) for NBD (see Lines 12–15). For time points after the decision (t > tRef ), Check NotBoth verifies that assigned values are the same as those at the time when the decision was made (at tRef ) (see Lines 16–20). In the Bike Lanes example, Check NotBoth would test the value of the NBN relationship between Permanent Construction and Temporary Construction. For every NotBoth relationship in the evolving goal graph, the time point loop iterates m = |Π| times. If we assume in the worse case that every relationship Chapter 4. Reasoning with Evolving Intentions 55

Algorithm 4.1 Check EI Func: Checks Evolving Functions for Intentions 1: function Check EI Func(EF , Π, aGoals) 2: for all item ∈ EF do 3: g = item.goal, 4: for all f ∈ item.function do 5: type = f.type, x = f.x, start = f.start, stop = f.stop 6: if type = Constant then . Test Constant function. 7: for all t ∈ Π ∧ t ∈ [start, stop) do 8: if aGoals(g, t) 6= null ∧ aGoals(g, t) 6= x then return false 9: else if type = Increase then . Test Increase function. 10: for all t ∈ Π ∧ t ∈ [start, stop) do 11: if aGoals(g, t) 6= null ∧ aGoals(g, t) > x then return false 12: for all c ∈ Π ∧ c ∈ [start, stop) do 13: if aGoals(g, t) 6= null ∧ aGoals(g, c) 6= null ∧ t > c ∧ aGoals(g, t) < aGoals(g, c) then 14: return false 15: else if type = Decrease then . Test Decrease function. 16: for all t ∈ Π ∧ t ∈ [start, stop) do 17: if aGoals(g, t) 6= null ∧ aGoals(g, t) < x then return false 18: for all c ∈ Π ∧ c ∈ [start, stop) do 19: if aGoals(g, t) 6= null ∧ aGoals(g, c) 6= null ∧ t > c ∧ aGoals(g, t) > aGoals(g, c) then 20: return false 21: return true in the graph is a NotBoth relationship, then the worst case runtime of Check NotBoth is O(n∗m). Graph Evaluation Path Algorithm (Algorithm 4.3). Algorithm 4.3 is a brute force algorithm that returns a complete graph evaluation path (satisfying Definition 4.5) or null indicating infeasibility. A complete graph evaluation path is a complete graph evaluation at each time point in the given time point path. Algorithm 4.3 can be used to answer stakeholder questions by generating simulations showing the evaluations of an evolving goal graph over time (see Section 4.3.3 for answers to questions from the Bike Lanes example). Algorithm 4.3 has four inputs: an evolving goal graph (satisfying Definition 3.11), a time point path (satisfying Definition 4.4), a set of user evaluations (satisfying Definition 4.2), and a set of conflicting evidence pairs (satisfying Definition 2.9). Algorithm 4.3 consists of two parts: initialization of the inputs (see Lines1–8) and the Solve function (see Lines 10–28). Solve is a recursive function that incrementally assigns evidence pairs to intentions at time points. When Solve is first called on Line9, it contains only the user evaluations in aGoals, and the remainder of the intentions in G are stored in uGoals with non-conflicting evidence pairs. For each iteration of the main loop (see Lines 17–27), Solve considers one additional assigned evidence pair. The new assignment, val for curGoal at curTime, is added to aGoals and removed from uGoals. Then, the Check EI Func (see Algorithm 4.1 for the Check EI Func function) and Check NotBoth (see Algorithm 4.2 for the Check NotBoth function) functions are called on Line 20 to determine if the assigned evidence pair violates any evolving functions. Next, the propagation loop (see Lines 21–23) iterates over each time point in Π. For each time point, the algorithm attempts to assign additional evidence pairs to intentions by calling the Propagate function on Line 22 (see Algo- rithm 2.3 for the Propagate function), and then checks that the assignment was correct by invoking the Check Prop function on Line 23 (see Algorithm 2.4 for the Check Prop function). Iterating over Chapter 4. Reasoning with Evolving Intentions 56

Algorithm 4.2 Check NotBoth: Checks Evidence Pair for Intentions with NotBoth Functions 1: function Check NotBoth(R, Π, aGoals) . Check NotBoth functions. 2: for all r ∈ R do 3: if r.type = NBN ∨r.type = NBD then 4: if r.type = NBN then fV al = (⊥, ⊥) . Determine required ﬁnal value. 5: else fV al = (⊥, F)

6: g1 = r.g1, g2 = r.g2, tRef = r.timeP oint . Initialize local variables. 7: for all t ∈ Π do 8: if t < tRef then . Test times prior to the decision. 9: for each g ∈ {g1, g2} do . Both intentions should be (⊥, ⊥). 10: if aGoals(g, t) 6= null then 11: if aGoals(g, t) 6= (⊥, ⊥) then return false

12: else if t = tRef then . Test decision time point. 13: if aGoals(g1, t) 6= null ∧ aGoals(g2, t) 6= null then . One value (F, ⊥), other fV al. 14: if ¬((aGoal(g1, t) = (F, ⊥) ∧ aGoals(g2, t) = fV al)∨ 15: (aGoal(g1, t) = fV al ∧ aGoals(g2, t) = (F, ⊥))) then return false

16: else if t > tRef then . Test times after the decision. 17: if aGoals(g1, tRef ) 6= null ∧ aGoals(g2, tRef ) 6= null then . Values same as at decision. 18: for each g ∈ {g1, g2} do 19: if aGoals(g, t) 6= null then 20: if aGoals(g, t) 6= aGoals(g, tRef ) then return false 21: return true each time point is required because Solve uses the untimed versions of Algorithm 2.3 and 2.4 from Chap- ter2. If any of these checks return false (meaning the graph is inconsistent), then the algorithm jumps to Line 26 to backtrack. If the checks all return true, then a recursive call to Solve is made with the additional assignment on Line 24. If the call to Solve returns null, then backtracking is required; otherwise, the call was successful and a set of aGoals is returned on Line 25. Solve backtracks by reverting aGoals and uGoals back to their old values and begins the next iteration of the main loop, where the next val is selected. If the main loop completes and all elements in curEvals had been tested, then no solution exists, and Solve returns null indicating infeasibility. Solve exhaustively checks every evidence pair for every unassigned intention at every time point, in the case of infeasibility. The above algorithm is naive. If one were to implement it, there are many ways of improving it, including caching and dynamic programming to store intermediate results as well as assigning evidence pairs to intentions based on evolving function values in advanced to limit the search space. Illustration of Algorithm 4.3. We can now ﬁnd a graph evaluation path for the Bike Lanes example. First, we ﬁnd a time point path given all the time points from the graph. The repeating User-Defined function for Bike Lane Usage has already been unrolled, so we have the following partial orders:

• t0 < tdesign < tdecision < (tuse = tBLOpen = r1(tuse))

• tuse < r1(tspring) < r1(tsummer) < r1(tfall) < (r1(tend) = r2(tuse)) < r2(tspring) < r2(tsummer) <

r2(tfall) < (r2(tend) = r3(tuse)) < r3(tspring) < r3(tsummer) < r3(tfall) < (r3(tend) = tend)

• tuse < ta1 < ta2 < tend

Table 4.2 lists the resulting time point path (note: tend is not included in the time point path, see Deﬁni- tion 4.4). The ﬁrst column lists the time points, and the second column lists the time evaluations for each time point. Then at each time point, we evaluate the goal graph given the constraints over the intentions. Using these time points and the goal graph as inputs to Algorithm 4.3, we can generate the remaining Chapter 4. Reasoning with Evolving Intentions 57

Algorithm 4.3 Compute Evolving Graph Evaluation Path Input: Goal Graph hA, G, R, EF , MC, maxTimei . Satisfying Definition 3.11. Time Point Path Π . Ordered set of times satisfying Definition 4.4. Set of User Evaluations UEval . Satisfying Definition 4.2. Set of Conflicting Evidence Pairs CFEvidence . Satisfying Definition 2.9. Output: Evaluations of the graph intentions (satisfying Definition 4.3) at each time point in Π or null indicating infeasibility.

1: aGoals = ∅ . Set of assigned intentions and evidence pairs. Type: hg, t, g i. 2: uGoals = ∅ . Set of unassigned intentions and possible evidence pairs. Type: hg, Jt, ∅iK . 3: for all t ∈ Π, g ∈ G do 4: if (g, t) ∈ UEval then 5: if UEval(g, t) ∈ CFEvidence then return null 6: aGoals.add(hg, t, UEval(g, t)i) 7: else 8: uGoals.add(hg, t, E\ CF Evidencei) . Add the set of non-conﬂicting evidence pairs. 9: return Solve(aGoals, uGoals, R, EF , Π)

10: function Solve(aGoals, uGoals, R, EF , Π) . Recursive procedure to pick values and check. 11: if uGoals = ∅ then 12: return aGoals 13: oldAGoals = aGoals, oldUGoals = uGoals . Create copies for backtracking. 14: curGoal = uGoals.head.goal . Get values at head of uGoals. 15: curTime = uGoals.head.time 16: curEvals = uGoals.head.evals 17: for all val ∈ curEvals do . Iterate over possible evaluations. 18: aGoals.add(curGoal, curTime, val) . Make assignment. 19: uGoals.remove(curGoal, curTime) 20: if ¬(Check EI Func(EF , Π, aGoals) ∧ Check NotBoth(R, Π, aGoals)) then goto Line 26 21: for all t ∈ Π do 22: (aGoals, uGoals) = Propagate(R.getT ime(t), aGoals, uGoals)) 23: if ¬Check Prop(R.getT ime(t), aGoals.getT ime(t)) then goto Line 26 24: result = Solve(aGoals, uGoals, R, EF ) . Call SOLVE to select the next assignment. 25: if result 6= null then return result . Return successful result. 26: aGoals = oldAGoals . Backtracking: restore previous assignments. 27: uGoals = oldUGoals 28: return null . If no evaluations remain, the graph is infeasible. Chapter 4. Reasoning with Evolving Intentions 58

Table 4.2: Graph evaluation path result for the Bike Lanes example. Each element of the time point column is associated with a time and an evidence pair for each goal in the goal graph. Notice that there is no row for tend/r3(tend) because the graph is not evaluated at this time point (see Deﬁnition 4.4). Time Points Time Bike Have Perm. Access Temp. Build Cyclist Have Lane Design Con- to Con- Bike Safety Bike Usage struct. Parking struct. Lanes Lanes

t0 0 (⊥, F) (⊥, F) (⊥, ⊥) (⊥, ⊥) (⊥, ⊥) (⊥, ⊥) (⊥, ⊥) (⊥, F)

tdesign 2 (⊥, F) (F, ⊥) (⊥, ⊥) (⊥, ⊥) (⊥, ⊥) (⊥, ⊥) (⊥, ⊥) (⊥, ⊥) tdecision 5 (⊥, F) (F, ⊥) (F, ⊥) (⊥, F) (⊥, ⊥) (F, ⊥) (F, ⊥) (F, ⊥) ta0 7 (⊥, F) (F, ⊥) (F, ⊥) (⊥, F) (⊥, ⊥) (F, ⊥) (F, ⊥) (F, ⊥)

tuse/tBLOpen/r1(tuse) 10 (⊥, P) (F, ⊥) (F, ⊥) (⊥, ⊥) (⊥, ⊥) (F, ⊥) (F, ⊥) (F, ⊥) r1(tspring) 13 (⊥, P) (F, ⊥) (F, ⊥) (P, ⊥) (⊥, ⊥) (F, ⊥) (F, ⊥) (F, ⊥)

ta1 14 (P, ⊥) (F, ⊥) (F, ⊥) (⊥, ⊥) (⊥, ⊥) (F, ⊥) (F, ⊥) (F, ⊥) r1(tsummer) 17 (F, ⊥) (F, ⊥) (F, ⊥) (P, ⊥) (⊥, ⊥) (F, ⊥) (F, ⊥) (F, ⊥)

r1(tfall) 19 (P, ⊥) (F, ⊥) (F, ⊥) (P, ⊥) (⊥, ⊥) (F, ⊥) (F, ⊥) (F, ⊥) r1(tend)/r2(tuse) 24 (⊥, P) (F, ⊥) (F, ⊥) (F, ⊥) (⊥, ⊥) (F, ⊥) (F, ⊥) (F, ⊥) r2(tspring) 28 (P, ⊥) (F, ⊥) (F, ⊥) (F, ⊥) (⊥, ⊥) (F, ⊥) (F, ⊥) (F, ⊥) r2(tsummer) 32 (F, ⊥) (F, ⊥) (F, ⊥) (F, ⊥) (⊥, ⊥) (F, ⊥) (F, ⊥) (F, ⊥)

ta2 34 (P, ⊥) (F, ⊥) (F, ⊥) (F, ⊥) (⊥, ⊥) (F, ⊥) (F, ⊥) (F, ⊥) r2(tfall) 36 (⊥, P) (F, ⊥) (F, ⊥) (F, ⊥) (⊥, ⊥) (F, ⊥) (F, ⊥) (F, ⊥) r2(tend)/r3(tuse) 40 (⊥, P) (F, ⊥) (F, ⊥) (P, ⊥) (⊥, ⊥) (F, ⊥) (F, ⊥) (F, ⊥) r3(tspring) 43 (P, ⊥) (F, ⊥) (F, ⊥) (P, ⊥) (⊥, ⊥) (F, ⊥) (F, ⊥) (F, ⊥) r3(tsummer) 45 (F, ⊥) (F, ⊥) (F, ⊥) (P, ⊥) (⊥, ⊥) (F, ⊥) (F, ⊥) (F, ⊥)

r3(tfall) 48 (P, ⊥) (F, ⊥) (F, ⊥) (P, ⊥) (⊥, ⊥) (F, ⊥) (F, ⊥) (F, ⊥) columns in Table 4.2, which give the complete goal graph evaluation at each time point. Time points tdecision and ta2 have already been discussed and illustrated in Figures 4.1 and 4.2. After tuse, only Access --S/NO to Parking and Bike Lane Usage change values. Once the Permanent Construction −−−−−→ Access to Parking tBLOpen link transitions at tBLOpen, Access to Parking is no longer the target of a propagation link and becomes a source intention with a stochastic value. Bike Lane Usage also changes because it is iterating through each interval in its User-Defined function. Correctness and Termination of Algorithm 4.3. In the remainder of this section, we prove properties of Algorithm 4.3.

Theorem 4.1 (Termination). Algorithm 4.3 terminates.

Proof. Let k be the size of uGoals at Line9 in Algorithm 4.3. The maximum number of possible evidence pairs is |E| = 9 for an intention at a given time, if no user evaluation is given (i.e., CFEvidence is the empty set). So each k has at most nine possible evidence pair assignments. Overall, Solve works by picking evidence pair assignments for intentions and backtracking when an assignment violates one or more of the constraints in the model. The recursive Solve procedure selects the tuple at the head of uGoals and then begins to iterate over possible evidence (curEval) for the selected intention (curGoal) at Line 17. If the selected assignment (val) does not violate any of the checks on Lines 20–23, then Solve is called again with the assignment held and a new head for uGoals on Line 24. If the assignment is not valid, then Solve will select the next evidence pair in curEval (Line 17) and repeat. If none of the values in curEval are found to be valid, then there is no valid solution, and Solve returns null. In the worst case, no additional values are found through the Propagate procedure, and Solve will try every combination of evidence pairs for every unassigned value, and will change the graph assignment Chapter 4. Reasoning with Evolving Intentions 59

9n∗m times (where n = |G| and m = |Π|, see discussion of Algorithm 4.1 above), before returning null. Therefore, Algorithm 4.3 is guaranteed to terminate with a runtime of O(9n∗m).

In practise there are smaller runtimes because the assignment of evolving functions reduces the search space, and additional values are found when calling the Propagate procedure, and when a CFLevel is given, there are fewer values to consider. We discuss actual runtimes in Section 6.1.2.

Theorem 4.2 (Completeness given a Complete Path). Algorithm 4.3 will result in a complete graph evaluation path if a complete path exists.

Proof. Proof is by contradiction: suppose a complete graph evaluation path exists and Algorithm 4.3 returns null indicating that the path is infeasible. The complete assignment is

{hg0, t0,(F, ⊥)i, ... hgn, tn,(⊥, F)i}. Since all user evaluations are known, and the Solve procedure tries every combination of evidence pairs for unassigned intentions, Algorithm 4.3 would have tried every combination, including this combination, which is a contradiction. Therefore, Algorithm 4.3 will result in a complete graph evaluation path if a complete path exists.

Theorem 4.3 (Soundness). If Algorithm 4.3 results in a complete graph evaluation path then the path satisﬁes Deﬁnition 4.5.

Proof. Definition 4.5 states that a complete graph evaluation path is the complete evaluation of the graph at each time point in the time point path, given the user evaluations and CFLevel. Definition 4.3 states that a complete evaluation of the graph at a time point is correct if all the user evaluations (at the time point) have their given values and all relationships satisfy the propagation rules in Figure 2.8 and the evolving functions (see Definition 3.2–3.10) such that ∀g ∈ G : g 6∈ CFLevel ∧ g 6= ⊥. J K J K If Algorithm 4.3 results in a complete evaluation of the goal graph, the final iteration of Check EI Func, Propagate, and Check Prop for each time point would have returned true so we know that the evaluation satisfies the rules in Figure 2.8 for all relationships in R, and Definition 3.2–3.10 for each time point. Given the user evaluations, the assignment of only non-conflicting evidence to unassigned intentions (see Lines3–8 in Algorithm 4.3) satisfies the requirement that ∀t ∈ P : G[t] → E such that ∀g ∈ G : g[t] 6∈ CFLevel ∧ g[t] 6= ⊥. Therefore, the complete evaluation of the goal graph is correct J K J K by construction. Chapter 4. Reasoning with Evolving Intentions 60

4.2 Reasoning in iStar

In this section, we discuss how reasoning with the Evolving Intentions framework can be adapted for iStar, and additional analysis we provide in iStar. As discussed in Section 3.2 in the context of modeling, there is nothing principally different about applying reasoning with the Evolving Intentions framework to iStar then what we presented for Tropos in Section 4.1. The same intuition carries for Definition 4.1 through Definition 4.5, as well as Al- gorithm 4.3 and the associated theorems. As discussed in Section 3.2, there are two key differences between Tropos and iStar: contribution and dependency link types, and the logic and propagation for evaluating intentions. This changes how each intention is evaluated at a time point with seven qualitative evaluation labels (see Section 2.1.3 for details), but again there is nothing principally different in iStar. However, since iStar does not have formal semantics, the results of Simulation over All Evolving Intentions do not have a unique interpretation (i.e., traceability), leading to confusion among modelers and stakeholders. For example, when a contribution link is resolved with forward analysis (see Table 2.3), it is not always known which source intention labels caused the destination intention label. The same issue exists with backward analysis. To further explore the merits of the Evolving Intentions framework in iStar, we developed a simulation approach based on forward analysis, where stakeholders assign evolutionary information to leaf intentions.

Compute Forward Graph Evaluation Path (Algorithm 4.4). Algorithm 4.4 creates a random path of the model given initial evaluation labels and evolving functions. It allows Stakeholders to answer questions of the form “what is the impact of choosing an alternative over time?” Algorithm 4.4 has three inputs: an evolving goal graph (satisfying Definition 3.11), a time point path (satisfying Definition 4.4), and a set of user evaluations (satisfying Definition 4.2). Unlike Algorithm 4.3, this algorithm finds the goal graph evaluation for each time point incrementally and then adds it to the path (pGoals). The main loop (see Lines2–8) iterates over each time point in the time point path. For the initial time point (t = 0), user evaluations are assigned to the model (see Lines4 and5). At each time point, the algorithm invokes the EF Eval on Line6, which considers the evolving function for each leaf intention (EF ) and the previous (or initial) evaluation of that intention. Once evaluations of leaf intentions are established, the algorithm propagates values to the root intentions on Line7 by calling the Label Graph function (see Algorithm 2.2 for details), and then adds the assignments in aGoals to the path (pGoals). Algorithm 4.4 returns a complete goal graph evaluation (satisfying Definition 4.3) for each time point in the time point path. We illustrate this process using the model fragment shown in Figure 4.3(a) for the Waste Management example, introduced in Section 1.2.1. The City wants to know if it can satisfy Manage City Waste over time. The City assigns evolving functions to three leaf-level nodes (shown in Figure 4.3(a)): Comply with Standards is assigned Constant (C ), Space in Dump is assigned Decrease (D), and Build Large Dump is assigned Denied-Satisfied (DS). Algorithm 4.4 takes the model (including these assignments) and a time point path as input, and returns one possible evolution of the model (see screenshots of the model fragments in Figure 4.3(b)). It iterates over each time point and returns the evaluation of the model at each time point. At time point 4, Space in Dump has the value PS, and at time point 10, Build Large Dump has transitioned to FS and Space in Dump has the value PD. Finally, at time point 12, Space in Dump has the value FD while Comply with Standards remains FS, and Manage City Waste has the value FS. Assuming the correctness of the functions EF Eval and Label Graph, Algorithm 4.4 is correct Chapter 4. Reasoning with Evolving Intentions 61

(a) Waste Management example fragment.

Time Step: 4 Time Step: 10 Time Step: 12

(b) EI-Sim results at time steps 4, 10 and 12.

Figure 4.3: Illustration of Algorithm 4.4 with the Waste Management example. by construction. The algorithm requires that evolving functions be assigned to leaf-level intentions and not be violated by user evaluations. Thus, the algorithm ﬁrst assigns constraints to leaf intentions and propagates the values throughout the model, resulting in a complete graph evaluation path. As discussed in Section 4.1.3, we let m be the number of time points in the Time Point Path Π, and n be the number of intentions in a goal graph (i.e., n = |G|). The runtime of Label Graph function is O(n4) (see discussion of Algorithm 2.2 for details). EF Eval has a runtime of O(n) because it iterates over each intention in the goal graph once. Since the main loop iterates m times for each element in the time point path, the worst case runtime of Algorithm 4.4 is O(n4 ∗ m).

4.3 Using Reasoning and Simulation Strategies to Answer Stake- holder Questions.

In this section, we introduce simulation strategies and describe how the questions are interpreted as strategies in the running examples.

4.3.1 Simulation Strategies

We allow stakeholders to examine alternatives over time through path-based strategies. Considering the time-based questions we have encountered, we adopt three simulation strategies: Chapter 4. Reasoning with Evolving Intentions 62

Algorithm 4.4 Computer Forward Graph Evaluation Path from Leaf Intentions Input: Goal Graph hA, G, R, EF , MC, maxTimei . Satisfying Definition 3.11. Time Point Path Π . Ordered set of times satisfying Definition 4.4. Set of Initial User Evaluations UEval . Satisfying Definition 4.2, where t = 0. Output: Evaluations of the graph intentions (satisfying Definition 4.3) at each time point in Π. Assumption: Only leaf-level intentions have functions in EF. Initial User Evaluations UEval do not violate evolving functions in EF.

1: pGoals = ∅ . Set of assignments in the path. Type: hg, t, g i. 2: for all t ∈ Π do J K 3: aGoals = ∅ . Set of assigned intentions and evaluation labels for this time point. Type: hg, t, g i. 4: if t = 0 then J K 5: for all (g, 0) ∈ UEval do aGoals.add(hg, 0, UEval(g, 0)i) . Add evaluations at t0. 6: aGoals = EF Eval(G, EF , MC, aGoals, pGoals, t) . Assign evaluations to leaf-level intentions based on Evolving Functions. 7: aGoals = Label Graph(hA, G, Ri, aGoals) . See Algorithm 2.2. 8: pGoals.add(aGoals) . Add intentions at t to the path. 9: return pGoals

ASR1 - create a random path given initial states and evolution information for leaf-level intentions in the model; ASR2 - create a random path given evolution information in the model and desired properties of any of the initial, intermediate, and final states of intentions; and ASR3 - create a path which is different than the previously seen path over the same constraints. ASR1. The first strategy is to create a random path given initial states in the model, with the goal of giving stakeholders a sense of model evolution over time. Modelers see one possible outcome of their model and can try different evolving functions for leaf-level goals. By running multiple simulations, modelers can see possible scenarios that may result, as well as experiment with different evolving functions for intentions to better understand how the intentions will become satisfied. This strategy creates a path starting from the initial state and is implemented by Algorithm 4.4. Given a state, we calculate the next state by randomly selecting intentions to update, given their evolving functions, and then check the resulting values against the model constraints, to ensure that the path is valid. Finally, we propagate the values throughout the goal model. If no values in the model can be further updated or all time points have values assigned, then the analysis returns the path. ASR2. The second strategy is to create a random path given evolution information in the model and desired properties of any of the initial, intermediate, and final states of intentions. This strategy allows stakeholders to set desired goals and see paths that satisfy those goals. Supplemented by Algorithm 4.3, this strategy produces a valid path when one is available. Depending on the constraints specified by the user, a valid path may not be possible. ASR3. The third strategy is to create a path which is different from the previously seen paths over the same constraints. Allowing stakeholders to construct new alternative paths enables them to understand other possible evolutions that exist. This is especially useful when dealing with stochastic variables and Chapter 4. Reasoning with Evolving Intentions 63

Motivating Example I: Waste Management Example (WME) - Modeler Questions WME-Q1 Is it feasible to first Build Green Centre and then build a dump (Build Large Dump or Build Small Dump)? WME-Q2 Does the order of these developments (Process Green Waste and Use New Dump) matter? WME-Q3 Which possible scenarios always satisfy Manage City Waste even if Space in Dump becomes denied in the future? WME-Q4 How do changes in Environmental Concern affect the city’s root-level goals over time? WME-Q5 Can all root-level intentions eventually be satisfied? WME-Q6 Given the initial results from answering WME-Q1-WME-Q5, is there another way to answer these questions?

Figure 4.4: List of modeler questions for the Waste Management example. Duplicated from Section 1.2.1 for ease of reference.

to understand the ordering of epochs. This strategy works the same way as ASR2 but has the added input of the previous path(s). The previous path’s time points are negated through additional constraints so that they are not picked again. If a new valid path exists, it is returned. Each intention gets a slightly diﬀerent negation constraint based on the intentions’ evolving function. For example, a Constant intention cannot be negated because it will break the constraints of the model. We discuss the implementation of these strategies in Chapter5.

4.3.2 Answering Questions in Motivating Example I: Waste Management Example (WME)

We first describe how each of these strategies can be applied to answer the Waste Management example questions posed in Section 1.2.1 (repeated in Figure 4.4 for convenience) with iStar. In order to understand the effect of building a green centre and then building a dump (WME-Q1), we need an exploration of the model, handled by ASR1. We can also use ASR1 to explore how changes in Environmental Concern affect the city’s root-level goals (WME-Q4). But in order to find simulations that always result in the future satisfaction of Manage City Waste (WME-Q3), we need to find a path given a desired final state, which can be handled by ASR2. Similarly, we use ASR2 when we want to find a path where all root-level intentions are satisfied in the final state (WME-Q5). ASR2 can also be used for intermediate states to test whether the ordering of events matters (WME-Q2). Finally, if we have an initial result and want to know if there is another answer to our questions (WME-Q6), then we need a strategy that takes previous results into account; this is handled by ASR3. ASR1: Path Given Evolving Leaf-Level Intentions. For ASR1, we can only add evolving functions for intentions to leaf-level intentions. In the WME model, the leaf-level intentions are:

{hBuild Small Dump, taski, hBuild Large Dump, taski, hBuild Green Centre, taski, hGW Education Program, taski, hHave Workers Union Contract, goali, hUpdate Truck Route, taski, hUpdate Truck, taski, hComply with Standards, goali, hPurchase Land, resourcei, hSpace in Dump, resourcei, hEnvironmental Concern, soft-goali, hSeparate Waste, taski}

We can use ASR1 to answer the question WME-Q1 (Is it feasible to first Build Green Centre and then build a dump Build Small Dump or Build Large Dump?), by initializing evolving functions and constraints. For each question discussed here we begin with the specification of the WME model in Figure 3.8, and keep A, G, R, and maxTime the same. To answer WME-Q1, we first construct new values for EF Chapter 4. Reasoning with Evolving Intentions 64 and MC.

EF = {hBuild Green Centre, hDenied-Satisfied, FS, {t0, tBGC , tend}ii,

hBuild Large Dump, hDenied-Satisfied, FS, {t0, tBLD, tend}ii,

hBuild Small Dump, hConstant, FD, t0, tendii,

hSpace in Dump, hConstant, FS, t0, tendii,

hUpdate Truck Route, hStochastic-Constant, FS, {t0, tUTR, tend}ii,

hUpgrade Truck, hStochastic-Constant, FS, {t0, tUTS, tend}ii,

hComply with Standards, hConstant-Stochastic, N, {t0, tCWS, tend}ii,

hEnvironmental Concern, hStochastic, N, t0, tendii,

hGW Education Program, {hConstant, FD, t0, tBGC i, hConstant, PS, tBGC , tSRP i,

hConstant, FS, tSRP , tERP i, hConstant, PS, tERP , tendi}i, hHave Workers Union Contract, hRepeat, 2, 0,

{hConstant, FS, t0, tEoC i, hConstant, FD, tEoC , tendi}ii}

MC = {tCWS = tMCR, tUTR < tSRP , tUTS < tSRP , tBGC < tBLD}

This model has thirteen unique time points:

t0 = r1(t0), tBGC , tBLD, tUTR, tUTS, tCWS, tSRP , tERP ,

r1(tEoC ), r1(tend) = r2(t0), r2(tEoC ), tMCR, tend = r2(tend)

These time points will form the time point path Π, with the exception of tend = r2(tend) which is not included (see Section 4.1.2). We also use a single user evaluation:

UEval = {hEnvironmental Concern, t0, FSi} Given these inputs, the strategy generates a path where Build Green Centre happens before Build Large Dump. This answer shows that it is feasible to ﬁrst Build Green Centre, and it results in the satisfaction of the root-level goal Manage City Waste. Similarly, we could have answered this question by using Build Small Dump instead of Build Large Dump, by updating the evolving functions as follows:

hBuild Large Dump, hConstant, FD, t0, tendii,

hBuild Small Dump, hDenied-Satisfied, FS, {t0, tBSD, tend}ii and adding the constraint tBGC < tBSD instead of tBGC < tBLD. This results in a simulation path, where Build Green Centre happens before Build Small Dump, and shows similar results.

We also answer WME-Q4 (How do changes in Environmental Concern affect the city’s root-level goals over time?) by initializing values for the leaf nodes and following this strategy. For WME-Q4, we can use the same inputs as we did above for WME-Q1 or those given in Figure 3.8. Environmental Concern affects the city’s root-level goals Reduce Operating Costs and Positive City Image through Process Green Waste. Stakeholders can see in the resulting path that when Environmental Concern becomes FD, Quality of Waste Separation is affected. Since Willingness to Separate Waste also has a Makes relationship with Quality of Waste Separation, Quality of Waste Separation becomes FD or U affecting Reduce Operating Costs and Positive City Image through propagation. It does not affect the other root-level goals, thus answering WME-Q4.

ASR2: Path Given Desired Intermediate Properties. WME-Q2 asks whether the order of Process Green Waste and Use New Dump matters. We generate three simulations to answer WME-Q2 by varying the constraint between Process Green Waste and Use New Dump (i.e., tP GW < tUND, tP GW = tUND, and tUND < tP GW ). This is implemented as a query feature in GrowingLeaf (see Section 5.1.1). We Chapter 4. Reasoning with Evolving Intentions 65 simpliﬁed the evolving functions for Comply with Standards and Have Workers Union Contract because they are not relevant to this question, and we have already demonstrated the unrolling of Have Workers Union Contract when discussing WME-Q1 above. We give the values for EF and MC required to answer WME-Q2.

EF = {hProcess Green Waste, hDenied-Satisfied, FS, {t0, tP GW , tend}ii,

hUpdate Truck Route, hStochastic-Constant, FS, {t0, tUTR, tend}ii,

hGW Education Program, {hConstant, FD, t0, tBGC i, hConstant, PS, tBGC , tSRP i,

hConstant, FS, tSRP , tERP i, hConstant, PS, tERP , tendi}i,

hComply with Standards, hConstant, FS, t0, tendii,

hHave Workers Union Contract, hConstant, FS, t0, tendii}

MC = {tUTR < tUND}

The solution where Use New Dump became FS first (via Build Small Dump) resulted in the model being satisfied. The solution where Process Green Waste became FS first resulted in Space in Dump becoming FD, which in turn resulted in Manage City Waste becoming FD. Answering WME-Q2, stakeholders learn that the order does not matter, but what does matter is the relationship between Use New Dump and Use Current Dump to ensure that Manage City Waste is always satisfied. Also, Space in Dump is not fully represented in the model yet.

Next, stakeholders ask WME-Q3: Which possible scenarios always satisfy Manage City Waste even if Space in Dump becomes denied in the future? To ﬁnd such a path, we model Manage City Waste as constantly satisﬁed and Space in Dump as a Monotonic Negative function. We add two additional functions to those used above for EF in WME-Q2:

hSpace in Dump, hMonotonic Negative, FD, {t0, tSiD, tend}ii,

hManage City Waste, hConstant, FS, t0, tendii. This strategy shows that one of Build Small Dump or Build Large Dump must be FS prior to the denial of Space in Dump. Since Build Small Dump results in the PS for Reduce Operating Costs and PD for Build Large Dump. Thus, Build Small Dump becomes slightly preferred.

Finally, we used this strategy to understand if all root-level intentions can eventually be satisﬁed (WME-Q5). We assign as input a set of evolving functions for root-level intentions in addition to other intentions:

EF = {hManage City Waste, hConstant, FS, t0, tendii,

hReduce Operating Costs, hStochastic-Constant, FS, {t0, tROC , tend}ii,

hPositive City Image, hStochastic-Constant, FS, {t0, tPCI , tend}ii,

hSpace in Dump, hMonotonic Negative, FD, {t0, tSiD, tend}ii,

hUpdate Truck Route, hStochastic-Constant, FS, {t0, tUTR, tend}ii,

hGW Education Program, {hConstant, FD, t0, tBGC i, hConstant, PS, tBGC , tSRP i,

hConstant, FS, tSRP , tERP i, hConstant, PS, tERP , tendi}i,

hComply with Standards, hConstant, FS, t0, tendii,

hHave Workers Union Contract, hConstant, FS, t0, tendii}

Analyzing these inputs resulted in no valid path because we included the requirement that Positive Chapter 4. Reasoning with Evolving Intentions 66

City Image and Reduce Operating Costs became FS, which is not possible given the model’s structure. Once relaxing these desired values to PS, as in:

hReduce Operating Costs, hStochastic-Constant, PS, {t0, tROC , tend}ii,

hPositive City Image, hStochastic-Constant, PS, {t0, tPCI , tend}ii, we did not find a solution that resulted in the PS for Reduce Operating Costs and Positive City Image at the same time. We could achieve one but not both, thus not all root-level goals can be satisfied. Stakeholders can use this information to evaluate priorities between root-level goals (i.e., Positive City Image and Reduce Operating Costs). ASR3: Path Given a Previous Path. We can use ASR3 to understand what other results (if any) exist as asked in WME-Q6: Given the initial results WME-Q1-WME-Q5, is there another way to answer the question? As we discussed WME-Q2 above, we noted that several different solutions existed for the ordering of Process Green Waste and Use New Dump. If we just consider the case where tUND < tP GW , the analysis selected a path where Use New Dump was satisfied by Build Small Dump. Using ASR3 with this result, we achieve an additional simulation result where Use New Dump was satisfied by Build Large Dump. This provides stakeholders with alternative solutions to whether or not each of Build Small Dump and Build Large Dump became satisfied (answering WME-Q6). Using ASR3 improves the modeling process because it removes the need for manual comparison of models when generating unique solutions. Summary. In Section 4.3.1, we introduced and mapped six questions from the Waste Management example onto our strategies (ASR1–ASR3). Using ASR1 on our example, we found that it would be feasible to build a green centre prior to building a dump (WME-Q1). We then used ASR2 to determine that the success is not affected by the order of these events (WME-Q2) but that either a small or a large dump must be built before the space in the current dump runs out (WME-Q3). By stochastically changing the citizens’ environmental concern in a random path simulation (ASR1), we were able to determine that it affects two root-level goals: (1) to have citizens enjoy the city, and (2) for the city to reduce its operating costs (WME-Q4). At this point, we used ASR2 to determine that all root-level goals could not be eventually satisfied (WME-Q5). Thus, stakeholders must make the tradeoff decisions between reducing operating costs and having their citizens enjoy the city, a typical trade off for many cities. We then used ASR3 to explore what other simulations could result when we process green waste prior to using a new dump, and concluded that it would be best to build a green centre followed by a small dump (WME-Q6).

4.3.3 Answering Questions in Motivating Example II: Bike Lanes Example (BLE)

We return to the Bike Lanes example to answer the modeling questions. In Section 3.1 and Section 4.1, we used a smaller fragment of the model to introduce the concepts of the Evolving Intentions framework. In this section, we use the full Bike Lanes Goal Graph, presented in Figure 1.2. In order to find a path where Access to Parking is satisfied at 6 months (BLE-Q4), we need to find a path given a desired intermediate state, which can be handled by ASR2. Similarly, we use ASR2 when we want to find a simulation that always results in the future satisfaction of Cyclist Safety (BLE-Q5). In order to understand the effect of putting the bike lanes or parking curb-side (BLE-Q1), we need an exploration of the model, handled by ASR1 or ASR2. We can also use ASR1 or ASR2 to explore the impact of Temporary Construction (BLE-Q2), and how changes in Bike Lane Usage affect the city’s goals (BLE-Q3). Using ASR1 restricts Chapter 4. Reasoning with Evolving Intentions 67 evolving functions to only leaf-level intentions; thus, we will use ASR2 instead so that we don’t have to consider this distinction. Finally, we can use ASR2 to explore deferring decision making (BLE-Q6). Since we use ASR2 for each question and BLE-Q6 has a different focus than the other questions, we first address BLE-Q6 and then explain how to answer BLE-Q4 in detail using ASR2, followed by a summary of the answers to the remainder of the questions.

BLE-Q6. Recall that in Section 4.3.2, whenever we chose between Build Large Dump and Build Small Dump in understanding the Waste Management example, we chose one of the options to consider in performing analysis and excluded the other option. In the Bike Lanes example, we let the reasoner choose between trade-offs. Using the NotBoth function, we identify mutually exclusive decisions and can ask if these decision points can be delayed (BLE-Q6). We begin by providing the partial specification (i.e., A, G, R, and maxTime) of the Bike Lanes example in Figure 4.5, where we added evolving relationships to the model. We specify EF and MC when describing modeling questions. Note that we added the --S/no relationship from Permanent Construction to Access to Parking as introduced in Section 3.1.3. The Bike Lanes example has two mutually exclusive decisions: Temporary Construction Plan vs. Per- manent Construction Plan1 for which we assigned an NBN function, and Bike Lane Curbeside vs. Parking Curbeside for which we assigned an NBD function (see Figure 4.5). By including these relationships in the goal graph specification, each generated simulation path will defer each decision and choose an alternative in the NotBoth relationship (see Definition 3.9); thus, we can delay decision making, answering BLE-Q6.

BLE-Q4. We use ASR2 to answer BLE-Q4: What scenario ensures Access to Parking is satisfied in 6 months, even if off-street lots become unavailable? The phrase ‘ensures’ in 6 months, means that Access to Parking must be satisfied at 6 months. We assign Access to Parking a Stochastic-Constant function

hAccess to Parking, hStochastic-Constant, FS, {t0, tacs, tend}ii, with the constraint tacs = 6. We give the values for EF and MC required to answer BLE-Q4.

EF = {hHave Design, hDenied-Satisfied,(F, ⊥), {t0, tdesign, tend}ii,

hBike Lane Usage, {hConstant,(⊥, F), t0, tusei, hRepeat, 3, 0,

{hStochastic,(⊥, ⊥), t0, tspringi, hIncrease,(F, ⊥), tspring, tsummeri,

hConstant,(F, ⊥), tsummer, tfalli, hDecrease,(⊥, F), tfall, tendi}i}i,

hNo Parking, hConstant,(⊥, F), t0, tendii }

MC = {tdesign < tplan, tuse = tBLOpen = 10, tacs = 6, tplan = 5, ta0 = 7, ta1 = 14, ta2 = 34}

The strategy returns a result where the Temporary Construction Plan is chosen, because otherwise the construction will not be completed by 6 months given the constraint that tplan = 5 (i.e., the time when a construction plan is chosen). This result helped us debug the model and realize that both the absolute values selected for Bike Lane Usage and the constraints within MC contained unrealistic values. To answer BLE-Q4 and ensure Access to Parking is satisﬁed in 6 months, the City must either select Temporary Construction Plan or make guarantees that permanent construction will be complete in 6 months. BLE-Q2. Looking more broadly now at the diﬀerences between temporary and permanent construction, we explore BLE-Q2: Is a permanent solution or a temporary solution most appropriate, given possible

1In Section 3.1 and Section 4.1, we used a fragment of the goal graph and this relationship applied to Temporary Construction and Permanent Construction instead. Chapter 4. Reasoning with Evolving Intentions 68

The Bike Lanes Goal Graph is hA, G, R, EF , MC, maxTimei where, A = {}, G = {hHave Bike Lanes, goali, hHave Design, goali, hBuild Bike Lanes, goali, hPlan Build, goali, hTemporary Construction, goali, hPermanent Construction, goali, hCommercial Access, goali, hCyclist Safety, goali, hAccess to Parking, goali, hBike Lane Usage, goali, hPublic Support, goali, hRoad Maintenance, goali, hOperate Bike Lane, goali, hRemove Bollards, goali, hNo Parking, goali, hSeparate Snow Removal, goali, hCosts Justification, goali, hConstruct Medians, goali, hInstall Bollards, goali, hPaint Lines, goali, hRoad Restructuring, goali, hOff Street Parking, goali, hTemporary Construction Plan, goali, hPermanent Construction Plan, goali, hParking, goali, hPrevent Dooring Incident, goali, hPrevent Unloading in Bike Lane, goali, hBike Lane Curbside, goali, hParking Curbside, goali}, and R = {(Have Design, Build Bike Lanes, Plan Build, Operate Bike Lane) −−−→ Have Bike Lanes, or (Temporary Construction, Permanent Construction) −−−→ Build Bike Lanes, or (Temporary Construction Plan, Permanent Construction Plan) −−−→ Plan Build, or (Remove Bollards, Separate Snow Removal) −−−→ Road Maintenance, or or (Bike Lane Curbside, Parking Curbside) −−−→ Parking,(Parking, No Parking) −−−→ Have Design, and (Paint Lines, Install Bollards) −−−→ Temporary Construction, and (Paint Lines, Road Restructuring, Construct Medians) −−−→ Permanent Construction, ++S --S Have Bike Lanes −−−→ Cyclist Safety, Separate Snow Removal −−−→ Costs Justification, -S + Remove Bollards −−−→ Costs Justification, Prevent Unloading in Bike Lane −−−→ Cyclist Safety, + + Bike Lane Curbside −−−→ Cyclist Safety, Prevent Dooring Incident −−−→ Cyclist Safety, + --S Parking Curbside −−−→ Prevent Unloading in Bike Lane, No Parking −−−→ Public Support, --S ++S No Parking −−−→ Access to Parking, Off Street Parking −−−→ Access to Parking, + + Access to Parking −−−→ Public Support, Commercial Access −−−→ Public Support, + ++ Bike Lane Usage −−−→ Public Support, Bike Lane Usage −−−→ Costs Justification, -S Temporary Construction Plan −−−→ Costs Justification, --S + Permanent Construction Plan −−−→ Costs Justification, Road Maintenance −−−→ Operate Bike Lane, --S/no Permanent Construction −−−−−→ Access to Parking, tBLOpen NBN Bike Lane Curbside ←−−→ Parking Curbside, tcurb NBD Temporary Construction Plan ←−−→ Permanent Construction Plan, tplan ++D Temporary Construction Plan −−−→ Temporary Construction, ++D Permanent Construction Plan −−−→ Permanent Construction}, maxTime = 50.

Figure 4.5: Speciﬁcation of the full Bike Lanes example goal graph with evolving elements. Chapter 4. Reasoning with Evolving Intentions 69

Motivating Example II: Bike Lanes Example (BLE) - Modeler Questions BLE-Q1 Are bike lanes or parking most effective on curb-side, given seasonal constraints on road operations? BLE-Q2 Is a permanent solution or a temporary solution most appropriate, given possible changes to cyclist and motor vehicle traffic? BLE-Q3 How do variations in the satisfaction of Bike Lane Usage over time affect the City’s goals. BLE-Q4 What scenario ensures Access to Parking is satisfied in 6 months, even if off-street lots become unavailable? BLE-Q5 Can we eventually satisfy and maintain Cyclist Safety? BLE-Q6 Can we delay the design and planning decisions to the future?

Figure 4.6: List of modeler questions for the Bike Lanes example. Duplicated from Section 1.2.2 for ease of reference.

changes to cyclist and motor vehicle traﬃc? We used the evolving functions EF listed above but removed all the absolute constraints in MC. In a simulation result for this question, we found another missing link between Install Bollards and Remove Bollards as well as between Construct Medians and Separate Snow Removal. Once these links have been added, the simulation indicated that Permanent Construction and permanent construction maintenance (via Separate Snow Removal) have a negative impact on Cost Justiﬁcation over the winter period when Bike Lane Usage is unknown (i.e., Stochastic). In the end, the City decides that Permanent Construction is deemed to be the better construction plan.

BLE-Q1. Another important tradeoﬀ is considered with BLE-Q1: Are bike lanes or parking most eﬀec- tive on curb-side, given seasonal constraints on road operations? Given the NBN relationship between Bike Lane Curbeside and Parking Curbeside, we can investigate this question with the same simulation results as BLE-Q4 and BLE-Q2, but this does not produce interesting results because we do not have a single representation of road operations and we have not added any evolving functions for the elements that impact Road Maintenance.

BLE-Q3. We investigate BLE-Q3: How do variations in the satisfaction of Bike Lane Usage over time aﬀect the City’s goals. We have already discussed the result of having Bike Lane Usage change with the season above, so here we change the evolving function for Bike Lane Usage to be stochastic,

hBike Lane Usage, {hStochastic,(⊥, ⊥), t0, tendi. We observe Bike Lane Usage changing stochastically in the resulting simulation path. This impacts Costs Justification and Public Support over the simulation. In this goal graph, Cyclist Safety is not impacted by Bike Lane Usage. The City must make a tradeoff decision between their root-level goals. If Cyclist Safety is paramount than it is recommended to find a solution where Cyclist Safety can be satisfied and maintained.

BLE-Q5. Finally, we look at BLE-Q5: Can we eventually satisfy and maintain Cyclist Safety? The phrase ‘eventually satisfy and maintain’ is modeled with an Stochastic-Constant function. So we add hCyclist Safety, hStochastic-Constant, FS, {t0, tsafe, tend}ii to the set EF in the goal graph. Us- ing ASR2 we generate a solution where cyclist safety has conflicting values until tsafe then becomes satisfied. We then modeled Cyclist Safety differently, using the function

hCyclist Safety, hIncrease, FS, t0, tendii, and added the user evaluations UEval = {hCyclist Safety, t0, PSi, hCyclist Safety, tsafe, FSi}. This combination will ensure that Cyclist Safety is always at least PS. But the model will not result in a complete solution because it cannot be partially satisﬁed at t = 0. We can eventually satisfy and maintain Cyclist Chapter 4. Reasoning with Evolving Intentions 70

Safety, but only if the bike lanes are built. Summary. By answering BLE-Q1–BLE-Q6, we explored the impact of changes in the evaluation of intentions in the Bike Lanes example. We concluded that the City should build the Bike Lane Curbeside with a Temporary Construction Plan.

4.4 Summary and Related Work

In this chapter, we described how to reason with Evolving Intentions. We formally defined how a goal graph evolves over time in Tropos. We first described how goal graphs are evaluated at specific time points and how these evaluations are combined to create a simulation path. We detailed a brute- force algorithm in Tropos for finding graph evaluation paths, proved properties of the algorithm, and illustrated the algorithm with the Bike Lanes example. We then commented on how the algorithm would apply for iStar and explained limitations to iStar’s informal propagation rules which lead to ambiguous results. We added a constant time algorithm in iStar for finding graph evaluation paths when evolution is restricted to leaf-level intentions. We also connected simulation generation with questions asked by stakeholders by establishing four simulation strategies and demonstrated how they can be used to answer the questions in the Bike Lanes example and Waste Management example. Finally, we answered the modeling questions proposed by stakeholders in Chapter1 for both examples. Related Work. Taken together, Chapter3 and this chapter gave the full details of the Evolving Intentions framework. As mentioned in Section 1.2.3, we are not the only ones to look at the evolution of evaluations in goal models. Aprajita et al. introduced TimedGRL, an approach that uses quantitative analysis to capture impacts of trade-offs at future time points [12][11][10]. If we consider the Bike Lanes example, TimedGRL can directly address BLE-Q3 and BLE-Q4 by specifying how the elements change in different contexts and then evaluating the model at each time point for a given context. TimedGRL partially supports answering other questions, such as BLE-Q1, and BLE-Q2, by creating different scenarios and manually comparing results. To fully answer the remaining questions, TimedGRL requires additional change types to determine values nondeterministically, for which is was not originally intended. However, TimedGRL has a better representation for the addition and deletion of intentions in the model with deactivation changes, and has additional model management features, which make it easier to test different evolving functions (i.e., dynamic contexts). In the broader methodology (i.e., TimedURN), GRL goal models are connected with UCM scenario models [11]. Rolland et al. also connected goals with scenarios for the purpose of automatically generating additional goals [123]. This work also provides a clear methodology for discovering goals. The Evolving Intentions framework can be extended to reason about scenario models, and can also be improved by having an explicit methodology to guide users in discussing evolution. The analysis presented in this chapter generates complete simulation results from t = 0. Nguyen et al. studied how to minimize total effort in finding incremental solutions when evolution occurs in the form of requirements changes [110, 111]. We have not investigated generation of incremental solutions based on previous or partial paths, but the analysis described in this chapter could be extended to find incremental solutions. Chapter 4. Reasoning with Evolving Intentions 71

Representation of Time. Our choice of representation allows for analyzing tradeoffs given changes in sociotechnical systems. Alternative approaches to the problem of analyzing change have been proposed. For example, as mentioned in Section 1.1.1, system dynamics is a methodology of scientific prediction used to model emergent properties of complex systems behaviour, where changes are nonlinear [43,138]. System dynamics looks at feedback (with causal-loop diagrams), as well as the accumulation of flows into stocks and time delays (with stock and flow diagrams) [138]. Causal-loop diagrams have been criticized for their simplicity and lack of semantics, and recommended to be used for teaching and explanatory writing for public consumption [120]. For example, causal-loop diagrams make no distinction between information links and rate-to-level links, and do not quantify impacts of sources, which can create misleading results. Causal-loop and stock and flow diagrams have been used to discuss global models and the impacts of geological limits on human society [105]. They have also been used broadly in organizational behaviour with case studies in business, engineering, and social and physical sciences [138]. In the Bike Lanes example, system dynamics would allow us to model traffic patterns of motorists and cyclists given different configurations, but would not allow us to make trade-off decisions in project planning. Creating systems dynamics simulations requires significant contextual information that is not usually known in the early-phases of RE. The intended purposes of goal modeling and systems dynamics are fundamentally orthogonal. Systems dynamics is suitable for simulating physical resources, but its applicability to this dissertation is limited. Existing work [148] already considered connecting goal modeling and systems dynamics. Zhang and Liu [148] mixed causal loop diagrams with basic goal models to refine top-level goals into causal loop relationships for modeling environmental impacts of systems. Future work could investigate connecting the Evolving Intentions framework with the work of Zhang and Liu. Choice of Logical Representation. In Section 3.1.1, we introduced our notion of time as ticks, time points and intervals between time points. This choice of representation was motivated by sequential logic from digital circuits and microprocessor design [101]. In earlier versions of this work, we used timing diagrams to represent changes in the fulfillment of intentions [58]. As we developed these ideas, we chose to use first-order logic, and prioritized the readability and simplicity of our representation over the expressive power. We also considered using LTL and CTL [13][28], as well as regular expressions [67]. Prior work in goal modeling used LTL for presenting temporal goals and timing properties [56][92]. LTL and CTL allow for the specification of constraints over paths and trees, respectively. These would not have been suitable for specifying Evolving Goal Graphs (see Definition 3.11), but would be suitable for specifying strategies over Graph Evaluation Paths (see Definition 4.5). For example, using ASR2, stakeholders consider properties over intermediate and final states of intentions, and could write these as LTL formulas where “eventually” an intention has a state. CTL could be used to express constraints over all possible simulations for a graph. For example, if stakeholders want to review all possible paths where an intention is eventually fulfilled. Future work could investigate using LTL and CTL as a way of automatically encoding strategies and stakeholder questions in order to check properties over simulations. Alternatively, we could imagine specifying evolving functions through a condensed text-based notation that would allow for the use of regular expressions to expand and create paths over individual intentions. This approach was not chosen because it required an additional level of abstraction but still used standard forward and backward analysis algorithms for propagation of labels. Part III

Tooling & Validation

72 Chapter 5

Tool Support

In this chapter, we describe how Evolving Intentions is implemented and we introduce two web-based tools we developed to evaluate the Evolving Intentions framework. We connect the algorithms and strategies with implementation details. GrowingLeaf was developed ﬁrst and allows stakeholders to model iStar goal models and answer questions using strategies ASR1–ASR3. BloomingLeaf was built by reengineering the code base of GrowingLeaf for modeling in Tropos. BloomingLeaf allows stakeholders to answer questions using strategy ASR2 and includes additional features (e.g., syntax checking).

Contributions

This chapter makes the following contributions. • We describe the implementation of the Evolving Intentions framework in GrowingLeaf for iStar and BloomingLeaf for Tropos. • We connect simulation strategies and components of the Evolving Intentions framework with implementation details. • We provide an overview of the architecture of each tool.

Acknowledgements. The tooling discussed in this chapter was developed in collaboration with under- graduate and visiting research students. We would like to acknowledge Jake Fear, Farhan Samir, and Gary Song for their contributions to GrowingLeaf, and Navie (Yikhei) Chan, Allen (Hanbin) Chang, Woran (Rose) Gao, Nasir Hemed, Boyue (Caroline) Hu, David Kwon, and Marcel Serikawa for their contributions to BloomingLeaf.

Organization

The remainder of this chapter is organized as follows: Section 5.1 introduces GrowingLeaf and gives an overview of the tool, its features, and architecture. Section 5.2 presents BloomingLeaf and provides a detailed overview of tool features, architecture, and ongoing work. Section 5.3 gives a sample procedure for generating models with evolution. Section 5.4 provides a summary and comparison of the tools with related work.

73 Chapter 5. Tool Support 74

Figure 5.1: Screenshot of GrowingLeaf with an iStar model of the Waste Management example.

5.1 GrowingLeaf

In this section, we introduce GrowingLeaf, the ﬁrst tool we implemented to evaluated the Evolving Intentions framework.

5.1.1 GrowingLeaf Overview

We introduce each part of GrowingLeaf using the Waste Management example (WME) introduced in Section 1.2.1. Modeling. GrowingLeaf allows users to build iStar strategic rationale models with actors, their intentions, dependencies, and intention links. The modeling view of GrowingLeaf is shown in Figure 5.1 with the WME model in the centre canvas. The left panel contains the stencil of model elements. Since Space in Dump is currently clicked in the model, its attributes are shown in the right panel including a chart demonstrating the behaviour of the Monotonic Negative function. Generally, users begin modeling by dragging actors and intentions (from the stencil on the left) onto the model canvas. When an intention is clicked, the right panel is populated and users can change the element’s name, qualitative satisfaction evaluation label, and dynamic function type (with a chart demonstrating the behaviour to make dynamic functions more intuitive). These updates are then shown on the canvas. Dependency and intention links are created by dragging the “>” icon from the source intention to the target intention. After selecting a link, its type can be changed in the right panel. Analysis. GrowingLeaf implements strategies ASR1–ASR3 (see Section 4.3.1). Figure 5.2 shows the analysis view of GrowingLeaf with a simulation result in the centre canvas. This simulation resulted from asking question WME-Q1, where we used ASR1 operationalized through Simulation over Evolving Leaf Intentions (implemented via the Leaf Simulate button in Figure 5.2). In the analysis view, the user clicks an analysis type from the right panel. The analysis type is then shown in the History Log and the resulting analysis is shown on the centre canvas as a series of models with a slider at the bottom allowing Chapter 5. Tool Support 75

Figure 5.2: Screenshot of GrowingLeaf analysis view.

the user to step through each time point. In the right panel, the user can also change the maximum number of simulation steps. The slider, which appears at the bottom of the canvas, indicates that the model is showing time step 20 of the simulation. The simulation is now listed in the History Log in the left panel. As discussed in Section 4.3.2, we used ASR2 to answer WME-Q2, WME-Q3, and WME-Q5. ASR2 is implemented with Simulation over All Evolving Intentions and encoded as a constraint satisfaction problem (CSP) (see Section C.1). Users can invoke ASR2 by selecting CSP Analysis in the right panel of the analysis view (see Figure 5.2). Finally, we used ASR3 to understand WME-Q6. This is also encoded as a CSP and implemented with the CSP History button in the analysis view of GrowingLeaf.

Additional Features. We encourage users to iteratively ask questions over their models and allow combining analysis results in the History Log. When an additional simulation is performed from the selected point on the slider, all satisfaction values up to the current time point are saved, and new satisfaction values are appended. Users can choose to merge previous analysis results forming a new sequence, by selecting the Merge Analysis button in the right panel (see Figure 5.2). At any point, the user can return to the modeling view from either the initial or currently viewed state of the model.

In the WME-Q1 simulation shown in Figure 5.2, we ensured that Build Large Dump became satisﬁed before Build Green Centre by adding a constraint in the constraint view. The constraint view looks similar to the modeling view, but with dependency and intention links removed to enable users to add constraints visually with directed edges. For example, Figure 5.3 shows the “<” relationship between Build Large Dump and Build Green Centre both on the model and in the right panel. We enable modelers to construct queries over constrains in the model. For example, to answer WME-Q2 (see Section 1.2.1), the modeler wants to know if the order between Process Green Waste and Use New Dump matters and if satisfying solutions exist for each order. We ask this question by selecting the intentions in the analysis view, which become highlighted in red (see Figure 5.4), and clicking Use Selected Intentions, shown in Figure 5.4, before clicking CSP Analysis. Chapter 5. Tool Support 76

Figure 5.3: Constraint view of GrowingLeaf showing the constraint between Build Large Dump and Build Green Centre.

Figure 5.4: Analysis view showing query function. Chapter 5. Tool Support 77

Figure 5.5: Architecture of GrowingLeaf.

5.1.2 GrowingLeaf Architecture

GrowingLeaf uses a simple client-server architecture. The front-end (client-side) is built using Javascript and HTML; the back-end (server-side) is implemented in Java. For analysis, we connect the front-end with the back-end through a Python CGI script, and an Apache HTTP Server. A high level overview of the architecture is shown in Figure 5.5. The front-end is built on top of Rappid, a modeling framework by client IO [31]. Rappid allows users to generate graphs using a drag and drop interface. Backbone is used to render customized side panels. In these side panels, GrowingLeaf allows users to specify constraints, relationships, and dynamics, as well as the type of analysis to be performed. Attributes specified in these side panels are stored in their corresponding objects in the model graph. The front-end handles all aspects of model creation and manipulation in order to eliminate the need for model storage on the server and reduce server calls. Our front-end extension is ∼13900 lines (written in JavaScript, HTML, CSS, and Python). The back-end analysis, consisting of ∼3800 lines of Java code, uses a modified version of jUCMNav’s systems architecture [6], and includes Java modules for producing both random simulation paths and creating constraint-based paths by interfacing with the JaCoP constraint solver [88]. When an analysis call is made by the client, all model objects are checked for appropriate syntax and then formatted into a text string (see AppendixA for a sample input file). A server call is made, passing this string with the pre-specified type of analysis, and storing it as a text file on our server. After the computation, the tool generates a results file with each model intention corresponding to a row of satisfaction values, one for each time point. The results file is returned to the client side for rendering – see Figure 5.2.

5.1.3 GrowingLeaf Discussion

Tool Usability. We completed two rounds of user testing with software engineering graduate students, with and without exposure to goal modeling. None of the students had a formal exposure to our techniques. In the first round, we found that students had difficulty resizing intentional elements and had issues with their intuitive expectations of the ‘enter’ and ‘backspace’/‘delete’ keyboard keys. They also had trouble understanding icons that appeared when they clicked on a canvas element. We have since Chapter 5. Tool Support 78 addressed many of these issues by disabling/simplifying elements. We did not have these same issues in the second round (with the exception of the ‘delete’ key problem); instead we found that students were not sure which analysis technique to use for the different questions they wanted to ask. We hope to mitigate this by adding descriptions of each analysis technique. See Chapter6 for our validation of GrowingLeaf. Design Decisions. Given the full array of iStar tools on the iStar wiki [1], we were hoping to add our analysis to an existing tool. We surveyed the available tools and reviewed a systematic comparison done in the literature [4]. None of the tools in active development appeared to have an easy to extend iStar meta-model and the ability to add labels on top of intentions. Since we chose to make our own tool, we decided to create a web-based tool. We first built a beta version of our tool to draw and evaluate goal models. We also introduced the concept of evolving functions to represent temporal changes in the satisfaction value of intentions. Our original design used a separate window for analysis results, but we observed that it was too taxing for users to switch between windows. When we added the constraints feature and investigated questions that took into account previous paths, we experimented with several layouts, ultimately settling on the one shown in Figure 5.2. Design Challenges. In developing our tool, we encountered two major challenges. Since we chose to build a web-based tool, the first challenge was dealing with the many browsers and their versions. Thus, we limited our development to ensure that our tool works in Google Chrome first. Although we have experimented with our tool on mobile platforms, we do not intend to support them at this time. When we wanted to add constraints between time points (see Figure 5.3), we weren’t able to show them on the canvas because the data model only contained a single type of link which we were using for dependency, decomposition, and contribution links. We had to revamp the architecture to add a second type of link. Given our substantial development effort prior to this discovery, we had to significantly refactor our codebase.

5.2 BloomingLeaf

In conducting the evaluation in Section 6.2, we found that the automatic propagation of iStar links resulted in confusing simulations for stakeholders. We then revamped our approach in terms of the formal syntax and semantics of Tropos. For this reason, we implemented a second tool, called BloomingLeaf, to validate the Evolving Intentions framework in Tropos. BloomingLeaf was developed from the initial code base of GrowingLeaf, and has the same look and feel. We kept the elements of the iStar language in the stencil of BloomingLeaf for more explicit visualization and discussions between modelers, although these elements are not considered in analysis. We also removed iStar dependency links, but users may choose to use the ++ contribution link to represent dependencies. The initial version of BloomingLeaf is complete and evaluation is ongoing. We introduce each part of BloomingLeaf using the Bike Lanes example (BLE), from Section 1.2.2.

5.2.1 BloomingLeaf Overview

Navigation. The interface consists of the stencil on the left, canvas in the centre, attribute panel on the right, and menu bar on the top (see Figure 5.7). The tool is made up of the Modeling View and the Analysis View. Users can identify their current view by the name in the top right corner of the canvas, Chapter 5. Tool Support 79

Figure 5.6: Model of Bike Lanes example in BloomingLeaf.

Figure 5.7: Modeling view with the goal Justify Cost selected. Chapter 5. Tool Support 80

(a) Goal attributes. (b) Link attributes.

Figure 5.8: Snippets showing the Attribute panel for element selected on the canvas, (a) shows the panel when Justify Cost is selected and (b) shows the panel when the link between Temporary Construction and Justify Cost is selected. as shown for the Modeling View in Figure 5.7. BloomingLeaf has additional features available in the top menu bar. For example, users can Save, Load, and Clear a model as well as export a model (i.e., Open as SVG) and change the font size.

Modeling. Goal models consist of goals and links forming directed acyclic graphs, conforming to the specification in Section 2.2. Goals can be decomposed into child goals requiring all (and) or one (or) child goal for satisfaction, but can also contribute to each other using contribution links (e.g., +, -, ++, --). For example, in Figure 5.6 Have Bike Lanes is and-decomposed into Build Bike Lanes and Have Design. Each goal can be evaluated using an evidence pair (s, d), where s ∈ {F, P, ⊥} is the level of evidence for and d ∈ {F, P, ⊥} is the level of evidence against the fulfillment of g (see Section 2.2). In Figure 5.6, Bike Lane Usage is given the value Denied (⊥, F) because the bike lanes are not currently in use. There is a ++S contribution link between Have Bike Lanes and Cyclist Safety. The ++S link only propagates when Have Bike Lanes has F or P for its s value. To add elements to the centre canvas, users drag them from the stencil on the left. Justify Cost was just added to the canvas in Figure 5.7. When the user clicks on Justify Cost the values for the goal are shown in the attribute panel on the right. The user enters “Justify Cost” for the Node name. The default Initial Satisfaction Value is None (⊥, ⊥) and the default Function Type is No Function. Suppose we want to eventually justify the cost. The Stochastic-Constant (RC ) evolving function specifies that after a time point t, the evaluation of a goal will be Satisfied (F, ⊥) and remain Constant. We can select RC as the Function Type with Satisfied (F, ⊥), as the constant value (see Section 3.1), and the panel will update (see screenshot in Figure 5.8(a)), with the chart now showing the updated function. When an element is clicked, for example, Justify Cost in Figure 5.8(a), the resize, delete, and link icons appear above the top-left corner of the goal. To create a link between two intentions, the user drags the “>” icon from the source intention to the target intention. By hovering over the link, a gear icon appears (see Figure 5.8(b)), and if the gear is clicked, the attribute panel on the right will show the information for the link. In Figure 5.8(b), the user selected a “+” link for the relationship between Temporary Construction and Justify Cost.

Analysis. Suppose we want to ﬁnd a path where Have Bike Lanes is satisﬁed (F, ⊥) at 36 months, in Chapter 5. Tool Support 81

Figure 5.9: Analysis view showing the analysis results at t = 36. order to answer the question “Which construction plan will satisfy the goals of interest in 36 months?” Have Bike Lanes was assigned the RC evolving intention function with Satisfied (F, ⊥) as the constant value. We then navigate to the Analysis View (see Figure 5.9). The attribute panel on the right now contains all the configuration options for analysis. BloomingLeaf implements strategy ASR2 with Simulation over All Evolving Intentions, and encodes paths as constraint satisfaction problems (see Section C.1). Users can invoke ASR2 by selecting Simulate Single Path in the right panel of the analysis view (see Figure 5.9). Since the analysis in BloomingLeaf can be mapped to absolute time points, we enter the Max Absolute Time (e.g., 48 in Figure 5.9, which maps to four years of wall clock time). We select Weak as our Conflict Prevention Level to avoid all conflicting values, and leave Num Relative Time Points as 0 because we are not interested in random sampling for this question. Finally, we add the values 6, 12, 18, 24, 30, 36, 42 to Absolute Time Points in order to sample the model every six months. To create a simulation, the user clicks Simulate Single Path, and a path where Have Bike Lanes is satisfied at 36 months is generated. The results will appear in the centre canvas with a slider at the bottom indicating the step in the path as well as the absolute time value (e.g., showing time step 7 with absolute time 36 for 36 months in Figure 5.9). At 36 months, we see that Temporary Construction was selected by the simulation and it has resulted in the satisfaction of Have Bike Lanes, Cyclist Safety, and Bike Lane Usage (see Figure 5.9). At this point, users carefully review the simulation results and propose alternative questions for analysis. By changing the evolving functions of goals and generating additional simulations, we can explore the best plan to satisfy Have Bike Lanes. Absolute and Relative Assignments. The Absolute and Relative Assignments window enables users to assign absolute time points to transitions in evolving functions, as well as specify the relative ordering of transitions in evolving intentions and relationships (see Figure 5.10). In the analysis above, we wanted the evolving function for Have Bike Lanes to transition to Satisfied (F, ⊥) by time point 36. To do this, the user clicks View Time Assignments and the Absolute and Relative Assignments pop-up appears. For example, in Figure 5.10 we entered “36” as the Assigned Time for the transition in Have Bike Lanes. Chapter 5. Tool Support 82

Figure 5.10: Absolute and relative assignments view.

Syntax Checking. The analysis in BloomingLeaf requires a model that is a directed acyclic graph and conforms to Deﬁnition 3.11. For this, we implemented cycle detection. If we add the link ++ Have Bike Lanes −−−→ Temporary Construction and click the pink Detect Cycles button in the top right corner, then an error pop-up window will appear (see Figure 5.12) and all elements involved in the cycle will be highlighted in red (see Figure 5.11). The red highlighting will disappear when the user breaks the cycle and re-runs cycle detection. Imagine instead that we created an or link between Have Design and Have Bike Lanes, but there was still an and link between Build Bike Lanes and Have Bike Lanes. This syntax is invalid because we require that each element be the destination of only one type of decomposition link. In this case, when the user clicks the green Analysis button, the syntax error pop-up message appears (see Figure 5.13).

5.2.2 BloomingLeaf Architecture

BloomingLeaf is a web-based tool consisting of a front-end on the client side and a back-end on the server side, as illustrated in Figure 5.14. The front-end is a dynamic webpage written in HTML and Javascript, which stores the goal model as a JSON string within the browser’s cookies. The live demo is hosted on our department’s Apache web-server (see http: // www. cs. utoronto. ca/ ~ amgrubb/ leaf-blooming-ui ). We built our interface on top of JointJS, a Javascript Diagramming Library, and use SweetAlert, ChartJS, noUiSlider, and KeyboardJS for interacting with and visualizing model elements. When a user makes an analysis request, the model and analysis conﬁgurations are packaged as a JSON string by the front-end and then passed to the back-end via the Common Gateway Interface (CGI). The back-end is a Java application that accepts an analysis request as a JSON string. The program uses Gson to convert the JSON string into Java objects (called Input Objects). The Input Objects are used to create Simulation Objects, which contain a collection of variables and constraints that form the constraint satisfaction problem. This collection is then passed to the Java Constraint Programming (JaCoP) solver, which returns a satisﬁable set of assignments to the variables, or an error. Chapter 5. Tool Support 83

Figure 5.11: Cycle detection feature highlighting cycle.

Figure 5.12: Cycle detection error message.

If the solution exists then the Simulation Objects are converted to a set of Output Objects. Gson is then used to convert the Output Objects back to a JSON string which is returned to the front-end for rendering. Excluding all external libraries, the tool is approximately ∼13000 lines of source code (written in JavaScript, HTML, CSS, and Python). Chapter 5. Tool Support 84

Figure 5.13: Syntax error message.

Figure 5.14: Architecture of BloomingLeaf. Chapter 5. Tool Support 85

5.3 Using the Evolving Intentions Framework

As discussed in Section 4.3, stakeholders can use the Evolving Intentions framework to answer timed- based questions. Prior to answering these questions, stakeholders must build and evaluate the completeness and validity of their models. In Section 5.1 and Section 5.2, we described how to use the modeling and reasoning capabilities of GrowingLeaf and BloomingLeaf, respectively. Here we give a brief overview of a guided procedure that enables stakeholders to generate and reason with goal models. In this process, the term ‘modeler’ refers to anyone who draws models or uses reasoning, which can include explicit stakeholders.

Step 1: Develop base model. Modelers should feel free to modify or add to the model in whatever way makes sense to them. Generally, they should begin by brainstorming all the actors they wish to model and adding them to the model. Next, modelers should focus on one actor at a time, and for each actor modelers should consider the following questions: • How does the actor interact with the other actors already in the model and those not yet in the model? • What goals motivate these interactions? • Can any of the intentions be decomposed? • Are there other ways of achieving the intentions of this actor? • Why does this stakeholder want to achieve this goal (discover other motivations)? Repeat this process with each actor (possibly multiple times) until all modelers believe that the model is complete.

Step 2: Add timing information. Identify the elements in the model whose evaluation (i.e., level of satisfaction or fulﬁllment) change over time. For each element consider three questions: • How do these evaluations change? • What assumptions do modelers have about this change? • What alternative patterns are there for each intention? Next, determine if there are any relationships that evolve over time. For each of these relationships consider the change, assumptions, and alternatives as posed above. At this stage the initial model is complete.

Step 3: Reasoning with the model. At this point, we recommend using the following reasoning loop to sequentially pose questions of the model: 1. Pose a question: Ask a question about the project. 2. Enrich the model: Ask the question, what evolving functions are required for this question? 3. Run Analysis: Connect stakeholder question to an analysis strategy. Run analysis and review/interpret the results. 4. Update the Model: Update the model after gaining some insight from the analysis. Ask the question, what would happen if an alternative evolving function was chosen for each of the intentions? 5. Repeat: Return to step 1, until satisﬁed with reasoning. We have not empirically validated this process, but developed it through our own work in the validation discussed in Chapter6 and Chapter7. At each step in this process, assumptions are made by models and can be documented. We do not currently support explicit documentation of these assumptions, but future work (as discussed in Section 8.2) may support this methodology further. Chapter 5. Tool Support 86

5.4 Summary and Related Work

In this chapter, we presented our implementation of the Evolving Intentions framework, in terms of two web-based tools (i.e., GrowingLeaf and BloomingLeaf). For each tool, we described the tool’s features and demonstrated their use with a motivating example. Table 5.1 reviews the features available in each tool. Notice the complementary nature of the tools, as neither tool implements the complete framework. We also gave an overview of the architecture for each tool. Work on BloomingLeaf is ongoing and we hope to add additional features. We are far from the first to implement a tool for goal modeling, and other tools are actively being developed (e.g., jUMC- Nav [7], Creative Leaf [71], MUSER [93], and piStar [117]). Furthermore, Almeida et al. completed systematic literature review of iStar tools [4], and Abad et al. performed a systematic literature review of visualizations in RE [3]. We are not the only ones working on tools for evolving goal models. An alternative approach, TimedURN, enables stakeholders to consider possible evolutions and trends through visualizations of quantitative goal models and feature models [12][99]. While we are unable to represent quantitative evaluations in GrowingLeaf and BloomingLeaf, with the addition of absolute time, BloomingLeaf can create simulations over models represented in TimedURN. We were not the first to use CSP for analyzing goal models. Luo and Amyot used CSP (and JaCoP [88]) to solve quantitative constraints in GRL [98]. We used this work to guide our implementation and extended it by enabling analysis over qualitative values. Other GORE researchers have used SAT to encode backward analysis for goal models [134][74], but we found the proposition-based encoding to be insufficient for our time-based analysis.

Table 5.1: List of features implemented in GrowingLeaf and BloomingLeaf.

Feature GrowingLeaf BloomingLeaf Relative Time Points X X Absolute Time Points X Atomic Functions X X Common Compound Functions X X User-Defined Functions X X Evolving Binary Relationships X X Evolving N-ary Relationships X X NotBoth Function X Graph Constraints (Relative) X X Graph Constraints (Absolute) X User Evaluations (Initial Time) X X User Evaluations (All Time Points) X ASR1 - Simulation over Evolving Leaf Intentions X ASR2 - Simulation over All Evolving Intentions X X ASR3 - Simulation over All Evolving Intentions with Path Histories X Syntax Checking X Chapter 6

Validating Evolving Intentions in iStar

In Chapter1 through Chapter4, we introduced the Evolving Intentions framework, an approach that includes a language for specifying how the evaluations of intentions change over time intervals. Once a model has Evolving Intentions, stakeholders can use Simulation over Evolving Leaf Intentions and Simulation over All Evolving Intentions to explore feasibility of particular scenarios, using simulation strategies ASR1–ASR3, described in Section 4.3.1. This analysis is realized in the tool GrowingLeaf (see a screenshot in Figure 6.3), which is described in Chapter5. In this chapter, we describe how we validated the Evolving Intentions framework using the iStar language. The initial exploration consists of evaluating the eﬀectiveness of the framework on a large example, and evaluating the scalability of the simulation strategies (ASR1–ASR3) using both Simulation over Evolving Leaf Intentions and Simulation over All Evolving Intentions. We then provide an addition validation of eﬀectiveness and usability of Evolving Intentions in a controlled experiment with graduate students. In the controlled experiment, we also study the general usability of GrowingLeaf, and the usability of Simulation over Evolving Leaf Intentions as a reasoning technique. See Chapter7 for further validation of Simulation over All Evolving Intentions.

Contributions

This chapter makes the following contributions. • We showcase the eﬀectiveness of modeling and reasoning with Evolving Intentions on a large example. • We demonstrate the scalability of reasoning on a variety of examples. • We present a controlled experiment, where we conclude that: - the representation of Evolving Intentions is suitable to the task of identifying and representing intentions over time, - Simulation over Evolving Leaf Intentions improves subjects’ ability to reason about goal models over time, and - the usability of GrowingLeaf is rated highly by subjects.

87 Chapter 6. Validating Evolving Intentions in iStar 88

Organization

The remainder of this chapter is organized as follows: Section 6.1 reports on the eﬀectiveness and scalability of our approach. Section 6.2 describes the controlled experiment. Section 6.3 provides a summary of the results.

6.1 Initial Validation: Eﬀectiveness and Scalability

In this section, we ﬁrst discuss the eﬀectiveness of our approach by analyzing a large example, and then we demonstrate how our reasoning techniques are scalable for larger models.

6.1.1 Eﬀectiveness: GrowingLeaf Analysis

In the absence of a real-world case study, we show our analysis with a goal model for developing our tool GrowingLeaf. GrowingLeaf was built to replace OpenOME [78] and we had already decided not to build it on top of other existing goal modeling tools. The primary functionality goals of the tool were to enable users to build goal models, run forward analysis over models, run time-based simulation over the models, and save/share the models between sessions. We wanted to evaluate alternatives for the tool based on five qualities: development speed, ease of installation, effectiveness, usability, and maintainability. We had to decide whether to build the tool as an Eclipse plugin or as an online tool in the browser. If we chose the Eclipse plugin option for development then we would be restricted to interacting with the Eclipse framework. If we chose the browser approach, we could completely separate the analysis from the front-end, but would need to have a way to connect them back. We modeled the intentionality of the developers and possible users of GrowingLeaf using iStar (not shown). After analyzing the models, we found that using Eclipse would be the best option given the expertise available in the group. Later, we discussed our plans with others and they told us of the difficulties of collaborating and extending Eclipse-based tools. In our previous analysis, we had not considered how our tool could evolve over time. Applying Evolving Intentions. An iStar model is shown in Figure 6.1 and has 9 actors, 74 intentions, and ∼100 links. We considered the motivations of the Collaborator and Lead Developer actors in the context of making the tradeoff decision about which web technologies to use for the project. We explicitly considered how evolution of the environment, development expertise, and relationships between stakeholders might change over time. We assigned evolving intentions and initial evaluations to some of the intentions in the model (see Figure 6.1). We evaluated themodel based on three options for technology choice, in which we and our collaborators (a) chose Eclipse, (b) chose Web-based technologies, and (c) chose other technologies. Figure 6.1 shows the scenario where we both picked Web-based technologies.

The Lead Dev had expertise in Java, specified as hConstant, FS, t0, tendi and modeled as C in Fig- ure 6.1, and had no expertise in web development but was willing to learn, modeled as MP and specified as hMonotonic Positive, FS, {t0, tl, tend}i. Eclipse development expertise was available through another developer who was willing to help for alternating intervals, specified as

hRepeat, 5, 0, {hConstant, FS, t0, tai, hConstant, FD, tb, tendi}i and modeled as UD. Each of the tasks in the Leaf actor can be speciﬁed with Denied-Satisfied functions (DS in Figure 6.1). Chapter 6. Validating Evolving Intentions in iStar 89

Figure 6.1: iStar model of GrowingLeaf example with selected analysis labels.

Analyzing the Model. Next we show that simulation produces meaningful results by discussing some of the questions we explored while developing the tool. Our first question is “If the Local Developer’s Eclipse Expertise has a repeated UD evolving function where they are available FS and then not available FD repeatedly, what will be the outcome of Build Tool and the root-level goals?” To understand this change over time we model the progression of the tool development given the variability of Eclipse Expertise with ASR1. We assigned initial values to leaf-level intentions and dynamics to intermediate nodes, for example applying Monotonic Positive function to Build UI with GMF. The resulting path showed a progression of satisfying Build Tool but was not able to satisfy all goals. This simulation showed us that establishing expertise in the project is required for it’s success, and our initial understanding using current approaches was incorrect. We are considering sharing our code/tool and wanted to know “What might happen if we don’t share the tool until after Complete PhD is satisfied?” This question is asking for a possible path given Chapter 6. Validating Evolving Intentions in iStar 90 intermediate constraints (ASR2). We add the constraint Complete PhD < Share Tool. In the resulting path, both Have Leaf Community and Have Lasting Impact do not have the value FS. Also, in paths where the Collaborator’s intention Use Eclipse was FD, the Collaborator’s root-level goals were denied. Thus, not sharing the tool until Complete PhD is satisfied is not favourable. We also looked at the question “In the long-term (identified as when Complete PhD becomes FS), we want Leaf’s Effectiveness, Usability, and Maintainability to be at least PS. What are possible solutions that will lead to these goals?” Given intermediate and final constraints, we first used strategy ASR2 to generate an initial path, and then used strategy ASR3 to generate additional unique paths. Seeing multiple paths showed that the order of completing some intentions did not matter, for example the satisfaction order of Have SD Models and Have Hybrid Models is irrelevant. Whereas, the order for completing other intentions mattered, specifically Share Tool should happen after Have Min Feature Set. As a result of this analysis we added further constraints to the model, in order to generate a PS outcome for Effectiveness and Usability in the long-term. Discussion. Asking these and other questions and considering the analysis results helped us debug the model and understand it. We found errors in the links and dynamics we initially assigned. More impor- tantly, doing time-based modeling clarified the potential dependencies between Lead Dev and Collaborator. We recognize that ‘research bias’ may be an issue in this effectiveness evaluation, because as researchers we were studying ourselves and we are biased towards the success of our technique. To mitigate this and ensure the accuracy of our models, we elicited feedback from other researchers and discussed the model in a workshop with our research group. There is a risk that the GrowingLeaf model may not be representative of other iStar goal models. We presented one fully-worked out example in this section, see Chapter7 for an additional example. Summary. We built this model to understand trade-offs in selecting development technologies, over our goals to have a functional, effective, useable, and maintainable tool. Standard analysis led us to the wrong decision because we assumed that the availability of Eclipse expertise is static. Our analysis allowed us to validate our decision to choose scenario (b): web-based technologies for development. The ability to generate multiple paths allowed us to determine which tasks must be completed in a prescribed order and which were independent. Overall, these strategies were effective in helping us understand possible evolutions of our project.

6.1.2 Scalability

In this section, we show the scalability of our approach by analyzing the run-times for each strategy (i.e., ASR1–ASR3) on the Waste Management example (see Section 1.2.1) and the GrowingLeaf Example (see Section 6.1.1) as well as randomly generated examples. We implemented ASR1 using Algorithm 4.4. We encoded ASR1 and ASR2 as a constraint satisfaction problem (CSP) (see Section C.1). In this section, we provide justification that using CSP as the underlying solver is feasible for realistic models. All scalability tests were completed on a 3.2 GHz Intel Core i5 processor with 8 GB 1600 MHz DDR3 of memory running OSX 10.11.5. Path Length of ASR1 and ASR2. Our first scalability experiment explored the question: “How does the length of the generated path affect the computation time in ASR1 and ASR2?” We explored this question with the Waste Management example (WME) consisting of 21 Intentions and 18 Links, and the GrowingLeaf Example (Tool) consisting of 74 intentions and 100 links. Each test was run ten times. Chapter 6. Validating Evolving Intentions in iStar 91

Figure 6.2(a) shows the mean run-times (in milliseconds) with min/max bars for WME and Tool for paths consisting of 5, 10, 25, 50, 75, 100, 150, 200, 300, 400, 500, 600, 700, 800, 900, and 1000 states. The maximum times for WME and Tool were 68 and 226 ms, respectively, and the shape of the graph illustrates that the run-time of AR1 appears to be growing polynomially with the length of the generated path. Figure 6.2(b) shows the mean run-times (in seconds) with min/max bars that strategy ASR2 took for WME and Tool for the same set of path lengths. The maximum times for WME and Tool were 13.52 and 91.89 sec which is significantly longer than ASR1. This may be a concern for larger models, with over 800 steps, because a run-time longer than one minute may cause the server call to timeout and the modeler to get frustrated. Further experiments with industrial models would help us understand and mitigate these limitations. ASR2 Graph Size. Given the lengthy run-times for ASR2 in the first experiment, our second experiment considered the question: “How does the number of intentions in a model affect the computation time in ASR2?” To evaluate variations in the number of intentions, we created a set of models by linking all intentions in a tree structure and varying the number of intentions in the models as follows: 25, 51, 75, 101, 125, 151, 175, and 201. For each model, we assigned an evolving function to a portion of the intentions. For each model, we calculated the mean run-time (in seconds) with min/max bars for generating paths of length 5, 10, 25, 50, 75, 100, 150, and 200. Figure 6.2(c) presents the results of this experiment. Some of the experiments initially failed due to run-time stack-overflow errors, requiring us to increase the default stack size. Figure 6.2(c) shows that model size does impact the run-time of our analysis and appears to have a greater effect than the size of the path. These experiments show that ASR2 (and CSP as the underlying solver) are expected to be scalable to realistic goal models. We have not attempted to optimize our CSP encoding to improve the run-time, but believe optimizations exist. Number of Previous Paths for ASR3. Since ASR3 generates a path based on previous paths, our final experiment asked: “How does the number of previous paths used affect the computation time in AR3?” It is not meaningful to evaluate the scalability based on the models generated for random evolving functions. Instead, we again used the WME and Tool examples to calculate the average runtimes to generate additional paths. In the experiment, we used path lengths of 15 and 30 states for each example, and generated three new paths given the previous paths. Figure 6.2(d) shows the mean results of the ASR3 experiment which was repeated five times for each new generated path. The maximum times for WME and Tool were 34 and 139 milliseconds, respectively, but these times were for a single previous path (where there was the most variability). The data suggests that additional previous paths reduce the run-time of AR3, making it more scalable than ASR2. We believe this is due to the additional constraints reducing the CSP search space (and thus its run-time). Summary. We conducted scalability experiments for our three strategies. For AR2, we used both case studies and generated models. From these results, we conclude that our strategies are scalable enough to be applied to the analysis of realistic goal models. Given that our scalability tests used randomly generated models, there may have been errors in the model or in the analysis. We mitigated this by manually reviewing the results for some of the generated models. Chapter 6. Validating Evolving Intentions in iStar 92

100 AR1 - WME AR2 - WME 200 AR1 - Tool 80 AR2 - Tool

100 40

0 Computation Time (Seconds) 0

Computation Time (Milliseconds) 0 200 400 600 800 1,000 0 200 400 600 800 1,000 Number of States in the Path Number of States in the Path (a) Results of changing the path length for ASR1. (b) Results of changing the path length for ASR2.

150 80 25 WME - 15 States 51 WME - 30 States 75 Tool - 15 States 60 100 101 Tool - 30 States 125 40 151 175 50 20 201

Computation Time (Seconds) 0 0 0 50 100 150 200 1 2 3 4 5 6 Number of States in the Path Average Comp. Time (Milliseconds) Number of Previous Paths (c) Results of changing the model size for ASR2. (d) Results of changing no. of prev. paths for ASR3.

Figure 6.2: Scalability tests for ASR1-ASR3. Chapter 6. Validating Evolving Intentions in iStar 93

6.2 Controlled Experiment: Eﬀectiveness and Usability

In this section, we investigate the eﬀectiveness and usability of Evolving Intentions (EIs), Simulation over Evolving Leaf Intentions (EI-Sim), and GrowingLeaf. We report on an ethics-reviewed between- subjects experiment conducted with ﬁfteen graduate students familiar with requirements engineering (RE). Our experiment aims to answer three research questions:

RQ 6.1 How do EIs aﬀect modelers’ ability to capture model elements that change over time? RQ 6.2 How does EI-Sim aﬀect modelers’ understanding and ability to reason about a goal model with time? RQ 6.3 How do modelers evaluate GrowingLeaf after completing modeling and analysis tasks?

The experiment is organized as follows: Section 6.2.1 describes the methodology, materials, and procedure for the experiment. Section 6.2.2 reports the results of the experiment. Section 6.2.3 discusses the validity and impact of the study. Section 6.2.4 compares our experimental methodology to related work.

6.2.1 Experiment Methodology

The goal of our study is to answer questions RQ 6.1- RQ 6.3. For each question, we describe the experimental design and materials required. The experiment design was developed and tested iteratively with research group members and received peer feedback at the IEEE 24th International Requirements Engineering Conference. This experiment was approved by the University of Toronto Research Ethics Board. See AppendixB for study protocol. Additional supplemental information is available online 1.

Experiment Design

To investigate our research questions, we need to evaluate subject cognition. Bloom’s taxonomy [86] (cognitive domain) defines six dimensions or levels of learning: remember, understand, and apply (ordered) followed by analyze, evaluate, create (parallel). We structure our questions to elicit cognition from a variety of levels (underlined) in Bloom’s taxonomy aiming for coverage across the levels. RQ 6.1. In asking RQ 6.1, we wanted to understand the effectiveness of EIs (introduced in Chapter3). We defined Stochastically Evolving Intentions (SEIs) as a control for comparison. SEIs change the evaluations of model intentions stochastically. From this, we generated four sub-questions: (i) How do subjects answer understanding questions about EI and SEI functions? (ii) How do EIs and SEIs affect subjects’ evaluation of changing intentions in the model? (iii) What representation would subjects create to indicate intentions with changing evaluations? (iv) How does priming with EIs and SEIs affect subjects’ evaluation of the model using forward analysis? RQ 6.2. In order to answer RQ 6.2, we wanted to compare EI-Sim, introduced in Section 4.2, with Simulation over Stochastically Evolving Intentions (SEI-Sim) and Repeated Forward Analysis (Rep-FA). SEI-Sim generates a new random value, independent of the previous value, for each leaf intention at each time step, and then performs forward analysis. Rep-FA does not actually generate a simulation; instead, we asked the subjects to repeatedly use forward analysis creating their own user-generated simulation manually and updating values at each iteration. Our null hypothesis was that there is no additional

1http://www.cs.toronto.edu/~amgrubb/archive/RE17-Supplement Chapter 6. Validating Evolving Intentions in iStar 94

Figure 6.3: GrowingLeaf modeling view with Waste Management model, used in RQ2, on the centre canvas. Functions shown are: Constant (C ), Decrease (D), and Denied-Satisﬁed (DS).

beneﬁt to using EI-Sim over SEI-Sim or Rep-FA, in the subjects’ ability to reason about the evaluation of intentions over time. We broke down this question by looking at how in the context of each analysis technique, the subjects would (i) identify (understand) and analyze alternative decisions, (ii) analyze the model given a changing intention, and (iii) evaluate the ordering for the completion of two tasks. RQ 6.3. In order to answer RQ 6.3, we asked the subjects to evaluate GrowingLeaf and provide con- structive feedback. We compared the evaluations for all study groups to see if there was a particular part of the tool that was better. RQ 6.0. In order to investigate RQ 6.1 and RQ 6.2, we had to teach the subjects EIs and/or SEIs resulting in a learning eﬀect. Thus, we evaluated all of our research questions between-subjects instead of within-subjects.

RQ 6.0 Do modelers perform similarly on basic cognition tests, given a consistent training protocol?

In order to answer RQ 6.0, the subjects were tested on their ability to answer a series of remembering and understanding questions about iStar goal modeling elements and forward propagation over links, as well as the deﬁnition of leaf and root nodes. Our null hypothesis was that the subject groups (see Section 6.2.1) performed equally well on the questions. We also tested the subjects’ ability to use the tool and apply forward analysis as a base line for completing the rest of the study.

Materials: Models, Tools, and Videos

We now introduce the materials we used in the study in order to ensure a consistent experience for each subject. Models. We used three models in this study. For RQ 6.0, we used the Trusted Computing model [73] (see Figure 6.4), which shows the relationships between a PC Product Provider, a PC User, and a Data Pirate Chapter 6. Validating Evolving Intentions in iStar 95

Figure 6.4: Trusted Computing model, used in RQ 6.1.

Table 6.1: GrowingLeaf tool versions created for the experiment.

Name Rationale Functionality GrowingLeaf-EI-Sim For focused learning of EI- Analysis view shows Forward Analysis and (Tool-EI) Sim and EIs. EI-Sim. No changes to the Modeling view. Analysis view shows Forward Analysis and GrowingLeaf-SEI-Sim Control for SEI-Sim to pre- SEI-Sim. Function Type selection removed (Tool-SEI) vent learning eﬀect of EIs. from Modeling view right panel. Intro version without EIs or Analysis view shows only Forward Analy- GrowingLeaf-Forward SEIs to prevent a learning sis. Function Type selection removed from Analysis (Tool-FA) eﬀect. Modeling view right panel.

surrounding the legal or pirated version of PC products. The Network Administrator model (see Fig- ure 6.5) was created for RQ 6.1 of this study. Its single actor, the Network Admin is considering which tasks should be completed and in what order, with the goal to Improve Network Infrastructure and In- crease Capacity. The third model shown in Figure 6.3, used in RQ 6.2, is an earlier version of the Waste Management example introduced in Section 1.2.1. Tools & Videos. We created three diﬀerent versions of GrowingLeaf and multiple training videos (with accompanying handouts) described in Table 6.1 and 6.2. To focus the subjects, functionality not needed by the study was removed from the UI in all tool versions. For example, Tool-EI was created so that subjects could use EI-Sim in isolation without exposure to SEI-Sim (row 1 of Table 6.1) and was introduced to the subjects using Video II EI (row 5 of Table 6.2).

Procedure: Conducting the Experiment

For our study, subjects were required to be graduate students with a basic understanding of requirements engineering and proﬁciency in English. Subjects were recruited through group mailing lists and an introductory graduate-level course in requirements engineering, and were oﬀered a chance to win a $50 Chapter 6. Validating Evolving Intentions in iStar 96

Figure 6.5: Network Administrator model, used in RQ 6.1.

Table 6.2: Videos created for the experiment.

Name Description RQ 6.0 Video 0A Reviewed goal modeling concepts/notations & introduced Tool-FA. Video 0B Introduced forward analysis with Tool-FA. RQ 6.1 Video I EI Introduced EIs. Video I SEI Introduced SEIs. RQ 6.2 Video II EI Introduced EI-Sim with Tool-EI. Video II SEI Introduced SEI-Sim with Tool-SEI. Video II AFA Introduced Rep-FA with Tool-FA.

gift certificate. Sixteen subjects volunteered for the study. One subject was unable to complete the study due to time constraints and their data has been excluded from this analysis. 9 Masters and 6 PhD students participated. The experiment was carried out in November 2016, one-on-one with the subjects in a meeting room. At the start, the subjects were asked to rate their familiarity with requirements engineering, and the iStar modeling language. Figure 6.6 shows the subjects’ responses: the majority were somewhat familiar with requirements engineering and not at all familiar with iStar. We did not have any subjects who believed they were extremely familiar (or experts) in the field. The subjects were randomly placed into one of four subject groups: Group A with five subjects (called A1-A5), Group B with five subjects (called B1-B5), and Group C, which was further divided into Group CA with three subjects (called CA1-CA3) and Group CB with two (called CB1 and CB2). The study procedure is listed chronologically in Table 6.3. For example, RQ 6.1 is discussed in the second line where Group A learned EIs by watching Video I EI and using Tool-EI, while Group B learned SEIs by watching Video I SEI and using Tool-SEI. Group CA and Group CB skipped this step to prevent a Chapter 6. Validating Evolving Intentions in iStar 97

Figure 6.6: The subjects’ self-reported level of familiarity with requirements engineering and the iS- tar modeling language. The shown percentages (left-to-right) refer to Not At All Familiar + Slightly Familiar, Somewhat Familiar, and Moderately Familiar + Extremely Familiar, respectively.

Table 6.3: Study procedure for each research question. Each triple consists of what the subjects learned/- did, which video they watched, and which tool they used.

Subject Groups Group C Group A Group B Group CA Group CB (n = 5) (n = 5) (n = 3) (n = 2) iStar & GrowingLeaf, Video 0A, Tool-FA RQ 6.0 Forward Analysis, Video 0B, Tool-FA EIs, SEIs, RQ 6.1 Video I EI, Video I SEI, skip Tool-EI Tool-SEI EI-Sim, SEI-Sim, Rep-FA, RQ 6.2 Video II EI, Video II SEI, Video II AFA, Tool-EI Tool-SEI Tool-FA EIs, SEIs, RQ 6.1 skip Video I EI, Video I SEI, Tool-EI Tool-SEI RQ 6.3 Tool Evaluation, N/A, N/A

learning eﬀect. Their completion of RQ 6.1 is listed in the fourth line of Table 6.3. We recorded answers electronically and documented questions asked by the subjects as well as novel uses of the tool/analysis.

6.2.2 Results

In this section, we describe results for RQ 6.0- RQ 6.3.

RQ 6.0

Remembering and Understanding iStar. To evaluate basic cognition, the subjects answered six questions, and their answers were scored out of 11 total correct answers (some questions asked them to identify more than one intention, e.g., “Name all the actors in the model?”). Figure 6.7 contains the bar chart for the subject scores, sorted by subject group. Nine subjects received a perfect score; three had one error; and one subject each received the scores 7–9 out of 11. Our null hypothesis was that the subject groups performed equally well on the questions. Using the Kruskal-Wallis Rank Sum (KWRS) test [127] we failed to reject this null hypothesis (p = 0.96) meaning that we could not detect a diﬀerence Chapter 6. Validating Evolving Intentions in iStar 98 between the groups. These results show that the subjects were able to successfully answer remembering and understanding questions, and are comparable.

Figure 6.7: Stacked bar chart of the subjects’ scores in RQ 6.0.

Applying Forward Analysis. The subjects were asked “If the PC Product Provider can only do one of Produce PC Products or Allow Peer to Peer Technology, which one is best for the top goals (use forward analysis to evaluate the alternatives)? Why?”. All subjects successfully applied forward analysis. Four subjects chose to satisfy Produce PC Products while eleven chose to satisfy Allow Peer-to-Peer Technology. Subjects who chose Allow Peer-to-Peer Technology gave the justification that it resulted in more root intentions becoming satisfied (more checkmarks). Of those who chose Produce PC Products, two required Abide By Licensing Regulations be satisfied; one focused only on PC Product Provider; and the other made an error remembering previous forward analysis results. Eleven subjects only looked at the analysis output when using forward analysis. The subjects were successful in the application of forward analysis. RQ 6.0 Completion Time. We evaluated the times the subjects took to answer questions, with the null hypothesis being that the subject groups answered questions in a similar length of time. Using the KWRS test, we again failed to reject this null hypothesis (p = 0.71), meaning that the subject groups were not significantly distinguishable in completion times for RQ 6.0. Summary. We conclude that the subjects performed similarly on remembering, understanding, and application tests, enabling us to compare the groups.

RQ 6.1

Understanding EI and SEI Functions. We asked “If Political Will has Stochastic (R) as its dynamic type and PD as its current evaluation, what possible evaluations will it have in the future?”. All Group A subjects and all but one Group B subject answered correctly. Group A was asked “If Develop Project has Monotonic Positive (MP) as its dynamic type and PD as its current evaluation, what possible evaluations will it have in the future?”. Understanding monotonic positive functions, four answered “PD, PS, FS” (i.e., the correct answer), and four answered “PS, FS”, assuming that the value must change between time points. This shows that the subjects understood the concept of EIs and SEIs. Evaluating EIs and SEIs. The subjects in both groups were asked to identify which intentions change over time, and specify how they change. Table 6.4 lists the recorded answers for each subject, with each row identifying each intention in the model in Figure 6.5, and each column listing whether each subject identified the intention as changing (blank if they didn’t) and their specification of that change. For example, Maintain Network was assigned a Constant (C ) function by A1. On average, Group A subjects identified two additional intentions as changing. Both groups identified primarily leaf Chapter 6. Validating Evolving Intentions in iStar 99 intentions (as expected), but most subjects believed that intermediate or root intentions changed as well. Group A subjects were able to identify EI functions by name. In describing how intentions change, Group B subjects identified many of the functions defined in EIs. For example, B1 identified Load Crisis as “a sudden change from satisfied to denied”, which describes the Satisfied-Denied (SD) function. B2 identified a new dynamic function where the value is stochastic (similar to R) but only between the values FS and FD. Subjects A3, A4, A5, and B3 wanted to assign a periodic function oscillating between two values. We conclude that both groups were able to evaluate changing intentions and there was little difference in priming with EIs or SEIs, when subjects were asked to refine SEIs. We also concluded that EI functions were intuitive, because Group B subjects identified EI functions in the absence of priming. Creating a Representation for Changing Intentions. We asked Group B “for the purpose of communicating with stakeholders how would you represent these dynamics symbolically in the model?”. Five subjects recommended using some form of a sparkline. Of those, two recommended using a single sparkline, two recommended showing probability distribution functions with the sparkline, and one recommended showing the intention information in a separate table. CB2 recommended adding a delta symbol to intentions that change, and CB1 recommended blurring the intention borders to differentiate between them. We found the majority of subjects created sparklines to identify changing intentions. GrowingLeaf’s design already uses sparklines to illustrate EI functions. Evaluating a Model using Forward Analysis with EI and SEI Priming. We asked subjects “if you can only do one of Update Current Technology or Increase Capacity2, which one is best for the top goals (use forward analysis to evaluate the alternatives)? Why?”. All subjects answered correctly (i.e., Increase Capacity). Three subjects stated the dominance of the Makes relationship between Have Sufficient Capacity and Increase Customers. Twelve subjects used trial and error by assigning various values (of which two did not assign values to other leaf intentions). Subjects A4 and B2 discussed changing intentions in the model, and A5 used EI functions. We conclude that although some subjects considered changing intentions, priming with EIs and SEIs did not impact the subjects’ model evaluations with forward analysis. RQ 6.1 Completion Times. Finally, we wanted to see if priming with EIs or SEIs affected the subjects’ RQ 6.1 completion times, as well as if there was a fatigue effect for Group C subjects who completed RQ 6.1 last. We tested the time between Group A and Group B, between Group A and Group CA, and between Group B and Group CB. For each of these three tests our null hypothesis was that there was no difference between the groups. Using the KWRS test, we failed to reject this null hypotheses: (p = 0.25) for Group A vs. Group B; (p = 0.88) for Group A vs. Group CA; (p = 0.053) for Group B vs. Group CB. Upon further investigation of our test of Group B vs. Group CB (which was barely significant), we found that both subjects in Group CB completed RQ 6.1 faster than Group B. We conclude there was no difference in times between priming with EIs or SEIs. If a fatigue effect existed in Group CB, it may correlate with the reduced number of changing intentions identified by this group. Model Changes. To elicit possible errors or threats to the analysis, we asked the subjects how they would change the Network Administrator model. Most recommended adding additional relationships to the model. Two recommended expanding Have Reliable Network, and another recommended removing Maintain Network. One subject thought that Improve Network Infrastructure should be a soft-goal while another thought that since it was a hard goal, it should be physically located above Increase Customers in

2While asking this question, the interviewee manually removed intentions to make Increase Capacity a leaf intention. Chapter 6. Validating Evolving Intentions in iStar 100 . ‘P’ FD and FS R R R R R R DS DS R SD UD RRRIRRRRRRCCRCCI F F FPF B1 B2DS B3SD B4RRRRRR B5 CB1 CB2 MP I R SD D MP R C R P P P P A? A A R RII ?A P P R MP I R RRRR I RRRRCRRRMP RMP MP I MP A1 A2? A3? A4 A5 CA1MP CA2 CA3 MP C? MP C R C CRR Position leaf leaf leaf leaf leaf intermediate leaf intermediate intermediate root intermediate root Elements Max Load Load Crisis Political Will Update Current Technology Maintain Network Get Capital Funding Develop Project Increase Capacity Have Reliable Network Improve Network Infrastructure Have Sufficient Capacity Increase Customers Table 6.4: SubjectRQ 6.1 . data for Entries forRQ 6.1 .specifies each a subject periodic The function list subjects’of that ‘F’, analysis. identifications oscillates ‘P’, between of ‘?’ ‘A’, two values. changing indicates ‘?’, ‘A’ intentions that or identifies in the an when subject the EI the either subject Network function. forgot determined Administrator ‘F’ to that model an identify defines intention a (see a changes function Figure only function or as that6.5 ) wasn’t the stochastically for sure. result changes Position between lists leaf & root intentions. Chapter 6. Validating Evolving Intentions in iStar 101 Willingness , X XXX XX X X Manage City Waste , XX X CA1 CA2 CA3 CB1XXXXX CB2 XXX CA1 CA2 CA3 CB1 CB2 XX (see Figure 6.3 ). X Build Large Dump , XX X X XXX X Environmental Concern Use New Dump B1 B2XXXXX B3XX*XX B4 B5 B1 B2 B3XX B4 B5 X X X XX X XX Table 6.5: Subject data for RQ 6.2 . A1 A2XXXXX A3XXX*X A4 A5 A1 A2 A3X A4 A5 was considered as a tradeoff with one of Positive City Image Use New Dump Produce Green Waste / / / ** Upgrade Trucks Use New Dump Build Large Dump / / / b The subjects’ responses to the impact of a change in the evaluation of , or itself. . ‘**’ indicates that Elements Build Small Dump Use Current Dump Build Green Centre Produce Green Waste All (leaf) intentions Space in Dump Comply with Standards GW Education Program Reduce Operating Costs Response Categories No Impact (Not much) One or Two Steps GraphStatic Trace Evaluation of Values Multi-Step Graph Trace Explanation of an Interaction Scenario Generation a Tradeoffs identified by theSpace subjects in Dump in theto Waste Separate Management Waste model (see Figure 6.3 ) for RQ 6.2 . ‘*’ indicates that the subject considered this tradeoff through Chapter 6. Validating Evolving Intentions in iStar 102 the model. These recommendations do not impact the results for RQ 6.1.

Summary. The subjects in both groups understood and evaluated EI and SEI functions, and evaluated intentions with them. SEI subjects created functions and graphical representations similar to EIs. EIs were found to be intuitive. Priming with EIs and SEIs did not impact forward analysis.

RQ 6.2

Understanding and Analyzing Tradeoffs with Simulation. We asked the subjects to “Identify all the alternative decisions in the model”, and then asked them to use their assigned analysis techniques to “choose the best alternative for each decision”. After, we asked them to “describe how the element evaluations vary over your analysis? Do any trends emerge?”. Table 6.5a lists the tradeoffs identified by the subjects. All subjects identified a tradeoff between Build Small Dump and Build Large Dump (first row), and most identified the tradeoff between Use Current Dump and Use New Dump. Five subjects incorrectly identified Build Green Centre and Upgrade Trucks as alternatives. Other alternatives discussed included Produce Green Waste compared with other intentions or itself, by five subjects. Individual items like Space In Dump and Comply with Standards were seen as tradeoffs themselves. B3 compared GW Education Program and Use New Dump, while C3 compared two soft-goals Reduce Operating Cost and Positive City Image. Three Group A subjects used EI-Sim by selecting dynamics for the model and obtained meaningful results. For example, A1 said “... [set] Space in Dump to Decrease (D) and now we have a problem. At some point, we will have an issue with the waste. When we have denied, we need to make sure the small or large dump is built”. Two subjects used only the Constant (C ) function, reducing the power of their simulations to forward analysis. Four out of five Group B subjects gained a better understanding of the model structure (via relationships) and considered the model as a whole. For example, B3 said “... Process Green Waste really needs a PS, because FS will not [satisfy] Positive City Image or Reduce Operating Cost”. The remaining subject focused on Manage City Waste to make decisions. Group C subjects used a divide and conquer approach to decision making, making one decision at a time. For leaf intentions not included in their current decision, they either applied FS or did not apply an evaluation label. For example, CA1 said “satisfying Use Current Dump would only satisfy Manage City Waste, but would not affect the decision for Build Small Dump or Build Large Dump”. We conclude that there is some effect between the use of simulation and model analysis with EI-Sim and SEI-Sim affecting the subjects’ understanding of the model structure. We have mixed results for the effectiveness of EI-Sim. EI-Sim affected the subjects’ understanding of time-based events, but two subjects did not use EIs. Rep-FA resulted in the subjects not considering the full model, making disjoint decisions instead. Priming with EIs and SEIs did not have an effect on how the subjects identified trade-offs. We learned that we need to better connect EI-Sim with EIs to improve the subjects’ usage of EI-Sim.

Analyzing a Single Changing Intention. We asked “What would be the impact if Environmental Concern changes in the future?”, and categorized each subject’s answer, listing the categories in the rows of Table 6.5b (ordered from worst to best). Responses that considered the impact in more depth were considered to be better. For example, answers that considered the propagation of multiple links were better than those that considered only one link, and considering possible evaluations of Environmental Concern and its impact on other interactions — better still. Two Group A and one Group B subjects generated scenarios for Environmental Concern. The best Group C answer was CB1 who explained the interaction as “it negates the eﬀorts of the GW Education Program. But you still get to Manage City Waste”. Chapter 6. Validating Evolving Intentions in iStar 103

The data in Table 6.5b signals that Group A and Group B had responses of more depth than Group C. This evidence was not significant but corroborates our previous result that simulation improved the subjects’ understanding of the model structure. Evaluating a Task Ordering Tradeoff with Simulation. We asked Group A/Group B [resp. Group C] the question “Assume you can sequentially complete both Build Green Centre and Build Small Dump. Which order is best for the top goals (use simulation [resp. forward analysis] to evaluate the alternatives)? Why?”. Three Group A subjects used EI-Sim and selected Build Small Dump, concluding that it was most important to Manage City Waste and that its satisfaction was dependent on Use Current Dump or Build Small Dump, and there were no guarantees that the current dump would last if the City completes Build Green Centre first. Those in Group A who did not use EIs (or just used constant values) selected Build Green Centre (or Process Green Waste). One made assumptions about the relative length of time it would take to build each item, and the other looked at the impacts on only two of the soft-goals. Group B unanimously chose Build Small Dump, but their justifications varied. Three subjects cited Build Small Dump’s contribution to Manage City Waste as the key reason. One subject stated a preference to satisfy the hard goals over the soft-goals. Another subject cited that Build Green Centre required Upgrade Trucks in order to satisfy Process Green Waste, whereas Build Small Dump directly impacted Manage City Waste and Reduce Operating Costs. Three Group C subjects chose Build Small Dump, concluding that since Build Green Centre does not affect Manage City Waste, Build Small Dump is preferable. These subjects did not consider the ordering; instead, they selected Build Small Dump assuming the City could only complete one alternative. CA1 selected Build Green Centre via Process Green Waste by evaluating the number of root intentions impacted by each alternative. CA2 concluded that “it doesn’t matter”. We conclude that EI-Sim enabled the evaluation of task ordering tradeoffs. EI-Sim and SEI-Sim helped the subjects understand the model structure. Using Rep-FA, the subjects did not consider time-based information. RQ 6.2 Completion Times. We tested to see if there was a difference in completion time between Group A, Group B, and Group C, to see if the simulation type affected RQ 6.2 completion times. Our null hypothesis was that there was no difference. Using the KWRS test, we failed to reject this hypothesis (p = 0.054), but since it was arguably significant, we ran Dunn’s test [38]. Dunn’s test performs post- hoc pair-wise comparisons between groups found significant with KWRS. We found that Group A was significantly different from Group B (Z = 2.3, p = 0.0098) and Group C (Z = 1.7, p = 0.045), but there was no significant difference between Group B and Group C (Z = −0.64, p = 0.26). Group A subjects took an average of almost six minutes longer to complete RQ 6.2 than the other two groups. We conclude that Group A’s completion times were significantly longer than Group B and Group C, which were not significantly distinguishable from each other. Model Changes. As with RQ 6.1, we elicited threats to the analysis by asking the subjects how they would change the Waste Management model. Multiple subjects recommended removing or connecting intentions that had no links (such as Purchase Land), and adding additional links. One also noted that Purchase Land was incorrectly modeled as a resource instead of a task. Other recommendations included quantifying the size of the dumps, making Environmental Concern more explicit, and changing the layout of the model. One subject found the dependum confusing. Finally, one subject in Group B thought that the model needed a long-term city waste management goal. These recommendations do not impact the results of RQ 6.2. Chapter 6. Validating Evolving Intentions in iStar 104

Figure 6.8: The subjects’ evaluation of GrowingLeaf. Likert graphs of the subjects’ level of satisfaction with modeling functionality, ease of use, appearance, and analysis functionality. The shown percentages (left-to-right) refer to Completely + Mostly + Somewhat Dissatisfied, Neither Satisfied or Dissatisfied, Somewhat + Mostly + Completely Satisfied, respectively.

Summary. EI-Sim and SEI-Sim improved the subjects’ understanding of the model structure, and EI-Sim improved the subjects’ ability to reason about goal models over time, but this analysis took signiﬁcantly longer. We learned that SEI-Sim, which was created as a control for comparison with EI-Sim, improved the subjects’ understanding of the model. The user-generated simulation (Rep-FA) proved diﬃcult for the subjects to answer time-focused questions.

RQ 6.3

Tool Improvements. We asked the subjects “What suggestions or changes would you recommend to the developers of this goal modeling tool?” and list their answers, grouped into required, desired, and other, in Table 6.63. Since the completion of the study, we have implemented all of the Required Improvements and are working on some of the Desired Improvements, all of which need some additional computational analysis to complete. Items listed as Other Recommendations are either outside the scope of our current research plan, have already been implemented but were not visible to the subjects, or are recommendations for the underlying language.

Tool Rating. We asked the subjects to rate the tool based on their level of satisfaction with ease of use, appearance, modeling functionality, and analysis functionality. Figure 6.8 contains the likert graphs of the subjects’ evaluations. Ease of use was evaluated the lowest, with 87% of the subjects satisfied and 13% unsatisfied. The subjects rated the appearance of the tool best, with 93% being satisfied and 7% being indifferent. The subjects rated the analysis functionality better than the modeling functionality. We also asked the subjects “How likely is it that you would recommend this goal modeling tool to a colleague?” The likert graph of the subjects’ responses is shown in Figure 6.9. All but one subject thought it was likely that they would recommend GrowingLeaf. We conclude that the subjects were overwhelmingly satisfied with the tool.

Tool Version Comparison. We examined if there was any difference in ratings across tool versions. Our null hypothesis was that there was no significant difference in how the subject groups rated the tool (ease of use, appearance, modeling functionality, analysis functionality, and likelihood to recommend the tool). Using the KWRS test, we failed to reject this hypothesis (p = 0.81, 0.83, 0.80, 0.24, 0.81 respectively), meaning that there was no discernible difference between the evaluations of the subject groups.

Summary. The subjects rated GrowingLeaf highly and found it usable.

3When the same suggestion was made by multiple subjects, we indicate it by ‘x2’ for two subjects and ‘x4’ for four subjects. Chapter 6. Validating Evolving Intentions in iStar 105

Table 6.6: GrowingLeaf improvements.

Required Improvements - Clear all intention evaluation labels. (x4) - Clear all dynamic function labels. - Disable delete key. - Allow intention names to span multiple lines. - Indicate whether something has changed in the previous step. - Add legend for the dynamic function labels and evaluation labels. - Change length of slider depending on the type of analysis. Make slider for forward analysis shorter (i.e., two steps wide). - Make PS/PD dots more obvious. - Create onscreen Help with instructions. Desired Improvements - Syntax checking while the user is modeling. - Highlight and unhighlight leaf intentions. (x2) - Highlight and unhighlight root intentions. - Overlay and or compare simulation paths. - Auto-resize model and intentions based on font size. - See steps within a forward analysis propagation. Other Recommendations - Change Depends arrows to make the arrow go in the opposite direction. (x2) - Make the shapes for goals and soft-goals more distinctive. - Prevent users from assigning dynamics to non-leaf intentions. (x2) - Create analysis where some values are ﬁxed and others vary. - Add a cost or utility function for each decision and then automate the analysis to ﬁgure out the optimal solution.

6.2.3 Discussion

This section discusses our statistical methods, followed by the broader implications and threats to validity of our study.

Statistical Methods

We use nonparametric statistics (specifically, the KWRS test) to evaluate if there are distinct group- ings within our sample data. Nonparametric statistics holds two main advantages over its parametric counterpart. First, for small sample size numerical data, nonparametric statistics avoids being unduly influenced by data points which differ greatly in magnitude. The completion time data benefited from this reduced sensitivity. Second, nonparametric statistics imposes no assumptions about the underlying shape of the probability distribution in its computation, and all our data benefited from this. Yet, our small sample size still limits the power of these tests. Chapter 6. Validating Evolving Intentions in iStar 106

Figure 6.9: Likert graph of the subjects’ likelihood to recommend GrowingLeaf. The shown percentages (left-to-right) refer to Extremely Unlikely + Unlikely, Neutral, Likely + Extremely Likely, respectively.

Implications for Research

The subjects were able to use EIs and EI-Sim, validating our approach for goal modeling over time, but their use was not perfect. The subjects missed the nuanced diﬀerences between the Monotonic Positive (MP) and Increase (I ) functions, and thought the value should change at each time point. We need to understand further why not all Group A subjects used EI-Sim eﬀectively, and better describe how EI-Sim depends on EIs. In some parts of the study the subjects paid closer attention to the content of the model than in others. We believe this to be a result of the lab setup and asking questions in isolation, but note that asking the “Why?” questions gave us access to how the subjects were thinking.

Implications for Education

Our experiment was done with iStar learners so it has implications for teaching. We used the dimensions of Bloom’s taxonomy to assess subjects cognition at multiple levels. We observed that subjects had diﬃculty with the Depends link and why it propagates information in the opposite direction of the link; the fact that the Breaks contribution link propagates a PS from FD; when to use the Unknown (U ) label; and which intentions to focus on in trade-oﬀ analysis. This is consistent with previous reports discussing students’ issues with Depends links and analysis [70]. Our study found that SEI-Sim helps subjects understand the structure of the model and how links propagate. Since SEI-Sim can be used without EIs, we believe it can be helpful in teaching propagation rules because learners see many possible analysis combinations and are able to ask questions about propagation results.

Threats to Validity

We discuss threats to validity using the categories in [144]. Conclusion Validity. Our main threat is our low sample size. With only fifteen subjects, the statistical power of any relevant statistical test will be low but we believe that the tests used were appropriate for our data (see Section 6.2.3). We automatically recorded completion times, starting immediately after each post-video discussion, to ensure reliable measurements. Data collection and analysis was made independent to reduce researcher bias. To mitigate reliability of treatment implementation, we standardized the experiment by maintaining the experiment setup throughout the study period, and used videos and handouts to ensure that the subjects had the equivalent training material (see Section 6.2.1). We do not believe there is a random heterogeneity of subjects risk in our study since our population was homogeneous, having similar knowledge, abilities, and previous experience with iStar and RE. Collecting addition demographics information would provide further evidence. Although researchers personally knew some of the subjects, they did not discuss the study with them prior to conducting it. Internal Validity. We used timing data and understanding questions to check for and mitigate against Chapter 6. Validating Evolving Intentions in iStar 107 selection-maturation interactions, where one group learns a treatment faster than another. We gave subjects time to review training materials to ensure they were ready to answer questions. We also checked for a maturation effect in RQ1, which Group C completed after RQ2, and we found an effect in the results of Group CB (discussed in Section 6.2.2). Since study participation was voluntary, a selection effect may exist because only motivated subjects participate. Further replication with an entire graduate class could mitigate this effect. To our knowledge, no subjects had used GrowingLeaf prior to the experiment, or iStar outside of course requirements. Construct Validity. This study was specifically designed to validate EIs as a construct and we feel they were accurately represented. In evaluating EI-Sim, we included both SEI-Sim and Rep-FA to evaluate whether there was a mono-operation bias because the SEI construct, by definition, is an under- represented version of EIs. We asked questions across dimensions of Bloom’s taxonomy to mitigate against mono-method bias. We mitigated experimenter expectancies by asking questions explicitly as worded in our protocol. As always, we have threats of hypothesis guessing and evaluation apprehension. Multiple subjects noted being nervous about the study because they were still quite novice. Additional studies evaluating these constructs can mitigate these threats. External Validity. Our homogeneous population (see Conclusion Validity) means that we cannot generalize our findings to the broader population of modelers. Further experiments with different populations, problem domains, and larger models for scalability are required to generalize these results. Our study was conducted one-on-one in a lab, which provides a foundation for further case studies, but does not directly generalize to early-phase requirements engineering done in “real” groups. We simulated the evolution of models in this study. Since our design was not longitudinal in nature, the subjects did not have the ability to witness changes in intention evaluations.

6.2.4 Related Work: Experiment Methodology

We built on the methodology of similar studies in RE for our between-subjects experiment and followed the guidance in [136, 144]. Karras et al. reported on a between-subjects lab experiment similar to ours [83]. Their treatment group had signiﬁcantly longer completion times than their control group, and we suspected that this was due, in part, to only one group learning a new tool. We attempted to control for this by giving the subjects exploration time with GrowingLeaf prior to asking them questions. Santos et al. reported a within-subjects quasi-experiment evaluating layouts of iStar models [132]. A within-subject experiment was not possible for our study due to a learning eﬀect. We removed the defect detection task from our study after considering the drawbacks discussed by Santos et al. and the similarities in our subject populations.

6.3 Summary

In this chapter, we validated the Evolving Intentions framework using the iStar language. We showed the application of the Evolving Intentions framework on the process of building our tool GrowingLeaf. We demonstrated the scalability of the analysis strategies ASR1–ASR3 and showed that using a CSP solver was appropriate for the task of analyzing goal models. We presented a between-subjects experiment to evaluate Evolving Intentions, Simulation over Evolving Leaf Intentions, and GrowingLeaf. We concluded that the representation of Evolving Intentions was suitable to the task of identifying and representing Chapter 6. Validating Evolving Intentions in iStar 108 intentions over time, that Simulation over Evolving Leaf Intentions improved the subjects’ ability to reason about goal models over time, and that GrowingLeaf was found to be eﬀective and usable. Chapter 7

Validating Evolving Intentions in Tropos

In this chapter, we aim to examine the applicability and effectiveness of the Evolving Intentions approach. Specifically, we study whether it can be used to simulate future evolutions of models where actor compositions and their intentions change. Case Scenario. Our intention was to find a well-documented example where sufficient evolution took place. In the absence of software engineering examples satisfying these criteria, we used historical data and rational reconstruction to examine how a controversial infrastructure project evolved in the past. The Spadina Expressway was a city planning and infrastructure development project in Toronto, Canada that spanned five decades. See Figure 7.1 for further exposition of the project scenario, and Figure 7.2 for a visualization of the route map. This project has been well studied and is a fitting example to evaluate Evolving Intentions because the project has a strong temporal component with many changes, including the introduction and withdrawal of stakeholders. Furthermore, over the course of the project, stakeholders who were initially opponents of the project became proponents, and vice versa. Investigation Criteria. Within the domain of the Spadina Expressway case, we used the following criteria to elicit evidence from the historical documents, and to specify our universe of discourse: IC1 supporters and opponents of the project, IC2 proposed highway configurations, IC3 proposed timelines for construction, and IC4 potential funders of the project. While not complete, we selected these for the availability of data based on a survey of the historical documentation (see Section 7.1 for further information). Model Creation with Varying Actors. Historical documents show significant changes in the actors over the course of this case with multiple actors existing over only part of the study. In our prior work, we assumed that all actors were present for the entire period of the project being modeled [60]. We needed a process for representing which actors are active at each time point. TimedGRL is able to represent the presence and absence of actors through Deactivation Changes [12]; but there is no literature on handling variance among actors for the creation of goal models. For the purpose of this study, we proposed

109 Chapter 7. Validating Evolving Intentions in Tropos 110 a process to generate a time-based goal model with varying actors. This process consists of creating scenario models and merging them into one model for the entire case. Research Questions. In this chapter, we present a rational reconstruction of the Spadina Expressway project in order to validate the modeling and reasoning capabilities of the Evolving Intentions approach. Rational reconstructions diﬀer from other case study methodologies in that they use historical real case data applied retroactively with the goal of demonstrating the eﬀectiveness of a technique or tool. Thus, we address a central research question:

RQ 7 To what extent is the Evolving Intentions framework expressive enough to capture the evolution of the Spadina Expressway project as documented in historical artifacts?

We break down this question into the following four speciﬁc research questions:

RQ 7.1 To what extent can we capture changes in the elements in our universe (deﬁned by IC1–IC4) with the framework? RQ 7.2 To what extent can the framework be used to reproduce the timeline of actual events? RQ 7.3 To what degree can the framework be used for generating and exploring ‘what if’ scenarios and alternative futures.

Contributions

This chapter makes the following contributions. • Presents a rational reconstruction of the Spadina Expressway project. • Validates the modeling and reasoning capabilities of the Evolving Intentions approach. • Presents a novel process for building time-based models based on merging scenarios that focus on groups of actors.

Organization

The remainder of this chapter is organized as follows: Section 7.1 describes the methodology for the rational reconstruction. Section 7.2 introduces each scenario. Section 7.3 explores simulating the scenario models. Section 7.4 deﬁnes the algorithm for merging evolving goal models and evaluates the full case model. Section 7.5 discusses study challenges and answers our overarching research question. Section 7.6 provides a summary of the results and connects our approach to related work.

7.1 Rational Reconstruction Methodology

We conducted a rational reconstruction of third degree data [39, 89], i.e., using archival data, on a real world descriptive case where the unit of analysis is the Spadina Expressway project itself. This rational reconstruction is unique among the requirements and modeling literature because we used multiple books written for other purposes [32, 114, 124, 135, 143], rather than a single requirements description. Each Chapter 7. Validating Evolving Intentions in Tropos 111

The Spadina Expressway was designed to connect Toronto’s Downtown core with the Macdonald- Cartier Freeway (Highway 401), the closest intercity controlled access highway. The Spadina Ex- pressway would also give downtown residents and Highway 401 users access to the Yorkdale Shop- ping Plaza (see Figure 7.2 for project illustration). It was part of a larger highway network first proposed in Toronto in the 1940s. The project was never fully completed and was finally closed to future development in the 1980s. Many have written about the project’s controversy and how this project led to a change in the structure of city planning. The expressway is an interesting case because it was instigated by politicians and planners in the downtown boroughs of the City of Toronto as a method of growth for the city and surrounding region. The adjacent suburbs opposed the project from the beginning, and fought against funding it. As economic development occurred with growth in both the suburbs and downtown, the project gained traction among the suburbs, while downtown residents associations became more opposed to the project given the perceived destruction on local neighbourhoods that had developed. The City of Toronto, who instigated the project, first lobbied the provincial government to force the suburbs to support the project through municipal restructuring. Then, when the suburbs supported the project, the city lobbied the province to stop the project and prevent the suburbs from completing it.

Figure 7.1: Spadina Expressway case description.

Yorkdale Plaza Crosstown Expressway Wilson Lawrence St. Clair Dupont Sussex Lakeshore Expressway

- North - Wilson-401 Lawrence Eglinton to Davenport - South - Bloor to Lakeshore Downsview Airport to Lawrence to Eglinton Davenport to Bloor Lake Ontario Highway 401 Eglinton Davenport Bloor Township of North York County of York (York) City of Toronto (Toronto) [Downtown] Municipality of Metropolitan Toronto (Metro)

Figure 7.2: Spadina Expressway route map. Illustrates the expressway segments from ‘Wilson-401 to Lawrence’ in the suburban North to ‘Bloor to Lakeshore’ in the downtown South, with markers for each cross street (italics identify interchanges with other expressways). The green-coloured segments were built, but the red segments were not. The legislative boundaries are shown below the map. intention and evolving function maps to one of the historical documents. Additional documentation including videos are available online1. In order to answer questions about the entire project (1947–1985), we chose to model groups of actors over three purposely chosen critical time periods in the project, and then merge them to create a single model for the entire history. This allowed us to answer questions about individual actors’ behaviour over shorter periods of time, and see how the answers to these questions diﬀer between models. We used the work of Runeson et al. [126], and Yin [145] as guidelines for conducting and reporting this study. Overview of Scenes. The key stakeholders of the Spadina Expressway project were the local governing bodies (i.e., City of Toronto (Toronto), County of York (York), and Municipality of Metropolitan Toronto (Metro)) and their planning boards, the Yorkdale Plaza project (Yorkdale Project), and local citizens and rate payers associations, collectively called the Stop Spadina Save Our City Coordinating Committee (SSSOCCC). We modeled three scenes to capture diﬀerent groups of stakeholders over explicit time

1http://www.cs.toronto.edu/~amgrubb/archive/Spadina-Supplement Chapter 7. Validating Evolving Intentions in Tropos 112 periods:

Scene I (1960–1964) looks at the project when it was being approved by the Municipality of Metropoli- tan Toronto (Metro), and the interactions with the Yorkdale Project.

Scene II (1970–1985) looks at the project when suﬃcient grassroots organizing had developed in an attempt to stop the project by the SSSOCCC, prior to the project’s cancellation.

Scene III (1947–1953) looks at when the Toronto and York Planning Board initially proposed the Spadina Expressway.

The three scene models were then merged to create the initial version of the Full Model (1947–1985), using the algorithm described in Section 7.4.1. Figure 7.3 illustrates the mapping between the timelines of each scene and the full case. We describe the context and scope of each scene using the investigation criteria IC1-IC4 introduced above. We then show how the scene is modeled and how changes within the scene are captured using our framework, which provides evidence for RQ 7.1. We then generate a simulation of historical events (to generate evidence for RQ 7.2) using the methodology and simulation inputs, as described in Chapter4. For the purpose of reconstructing the historical timeline, the passage of time is observed in six-month increments, with time points of January 1 and July 1 of each year. The timeline is then mapped to simulation times from t0 = 0 to the maxTime, as illustrated in Figure 7.3. Finally, we consider one alternative future (to collect evidence for RQ 7.3) for each scene. Table 7.1 lists questions that we posed in order to generate alternative futures. We created additional simulations by altering some of the simulation inputs used for reproducing the historic events and generated a simulation of alternative events. In this chapter, we only present fragments of each scene model. The full models and specifications for each simulation are available online1. Model Scope and Assumptions. This study was completed using BloomingLeaf (see Chapter5). We scoped each model to include what was minimally needed for answering our research questions. For example, we modeled the Network Project’s goal to Have a Unified Arterial Road System, but we did not model whether cars were inherently good or the environmental impacts of building a road system. We also abstracted away the details of the other road projects in the network to focus on the Spadina project. We limited our universe to actors mentioned in the historical documents, and did not consider any events prior to 1947 and after 1985. In building these models, we made several assumptions resulting in specific modeling decisions. We did not model the explicit costs of the project, but did explicitly consider which stakeholders would agree to pay for the highways. As we discuss in Section 7.2, we modeled Get Funding From Toronto to represent the City of Toronto’s commitment to funding the Spadina Expressway project.

7.2 Modeling Scenes

In this section, we introduce each scene and describe the model we created using the historical documents. We review relevant background from Section 2.2 and Section 3.1 as needed in order to contextualize each model in Tropos. Chapter 7. Validating Evolving Intentions in Tropos 113

M maxTime FULL 0 12 25 33 45 76 times (t) Time: 1947/7 1953/7 1960 1964 1970 1985/7 Scenes 0 12 0 8 0 31 times (t) III(MPLAN ) I(MPRO ) II(MOPP ) 0 51 MIM (Section 7.4.1)

Figure 7.3: Mapping between wall-clock and simulation time, for each model.

7.2.1 Scene I (1960–1964)

In Scene I, we focus on the proponents of the Spadina Project. At this point, the project was under the authority of the Municipality of Metropolitan Toronto (Metro), but was dependent on the Yorkdale Project, York, and the Province of Ontario (Province). Figure 7.4 shows the MPRO model, a simplified fragment of the Scene I model and contains these five actors with their intentions within the dotted actor boundary (investigation criterion IC1). The central goal of the Spadina Project is to Have Spadina Expressway. This goal is and-decomposed into the tasks Plan Project, Get Funding, and Build Spadina Expressway, meaning that all three are required for the satisfaction of Have Spadina Expressway. In this period, Metro had considered two different configurations of the Spadina Expressway (IC2): the Connected Expressway which would connect with the Crosstown and Lakeshore Expressways; and the Terminal Expressway which would transition to a full access road (i.e., Spadina Avenue) at Bloor Street. In Figure 7.4, Build Spadina Expressway is or-decomposed into these two configuration trade-offs; only one of them needs to be satisfied to satisfy Build Spadina Expressway. Connected Expressway and Terminal Expressway are then and-decomposed into the road segments required for each plan. Most segments are required for both, but Build Bloor to Lakeshore and Build Crosstown Interchange are only required for the connected plan (see Figure 7.2 for the segment map). Recall from Section 2.2.2, each intention can be evaluated using an evidence pair (s, d), where s is the level of evidence for and d is the level of evidence against the fulfillment of the intention. In the figures in this chapter, we use the following symbols to represent these evidence pairs: FS = (F, ⊥), PS = (P, ⊥), PD = (⊥, P), FD = (⊥, F), and N = (⊥, ⊥). For example, Build Bloor to Lakeshore is assigned FD in Figure 7.4, meaning (⊥, F) (no evidence for and full evidence against), as this part of the road was not built. The Yorkdale Project has the goal Have Yorkdale Shopping Plaza, proposing to build a large shopping plaza at the junction between the proposed Spadina Expressway and the Highway 401. Plan for Shopping Plaza is assigned FS because the plan for the shopping plaza has already been released. The Yorkdale Project needs access to Highway 401 (e.g., modeled as the goal Have access to 401). There is a ++ link from Build Wilson-401 to Lawrence to Have access to 401 meaning that if Build Wilson-401 to Lawrence is satisfied, then Have access to 401 will be satisfied. In fact, the ++ link propagates both evidence for and evidence against the fulfillment of the intention. The +S link from Have Yorkdale Shopping Plaza to the Economic Development soft-goal in the Province actor means that if the Have Yorkdale Shopping Plaza is satisfied, then there exists partial evidence that Economic Development is also satisfied. Scene I models the approval and initial construction of the project between 1960 and 1964 (maxTime = Chapter 7. Validating Evolving Intentions in Tropos 114

Figure 7.4: MPRO : Scene I fragment at t = 4 (Jan. 1962).

8, see Figure 7.3). We specify how the evaluation (i.e., an evidence pair) of an intention changes over time with evolving functions. Recall from Section 3.1.2, an evolving function is a sequence of these atomic functions (i.e., Constant, Increase, Decrease, and Stochastic) over disjoint neighbouring intervals. For example, Plan for Shopping Plaza remains Satisfied (FS), so we assign a single Constant function over the entire period of Scene I, identified by the C label on Plan for Shopping Plaza in Figure 7.4. York was still opposed to both configurations as planned because it meant disrupting a local community and the Cedarvale Valley, and refused to release the land required for the Spadina Expressway. York’s Approve Land Release task is assigned a Constant function remaining Denied. It was decided that Metro would pay for half of the Spadina Project, and the Province would pay for the other half (investigation criterion IC4), modeled as the and-decomposition of Get Funding into Get Funding From Metro and Get Funding From Province. The Yorkdale Project was pressuring Metro to approve the project and start construction. The Province, motivated by their desire to Support Economic Development, wouldn’t fund the project until a plan was approved by Metro; thus, Metro satisfied Approve Plan and Approve Construction to Lawrence, despite the land conflict with York. We have identified functions that occur commonly over two time intervals (see Table 3.2 for a full list). For example, Denied-Satisfied (DS) specifies that the evidence pair assignment of an intention remains Denied (⊥, F) over one interval and then remains Satisfied (F, ⊥) over a second interval. Approve Plan, Approve Construction to Lawrence, Approve Funds, and Approve Project Funding follow this pattern, and are all assigned DS labels in MPRO (see Figure 7.4). Metro began building the segment that connects the Highway 401 and the Yorkdale Shopping Plaza Build Wilson-401 to Lawrence is assigned a unique evolving function, called a User-Defined (UD) function, where the value is first Denied (⊥, F) then increases between Jan. 1964 and July 1966, and Chapter 7. Validating Evolving Intentions in Tropos 115

Figure 7.5: MOPP : Scene II fragment at t = 12 (Jan. 1976). then remains Satisﬁed (F, ⊥) to model construction activities. This evolving function gives a construction timeline (investigation criterion IC3), in the form of modeling constraints. MPRO = hA, G, R, EF , MC, maxTimei is an example of an evolving goal graph (see Deﬁnition 3.11). When referring to elements of models used in this chapter, such as MPRO , we refer to its components using the dot notation, so MPRO .G is the set of intentions that belongs to MPRO .

7.2.2 Scene II (1970–1985)

Scene II (see MOPP in Figure 7.5) examines the opposition to the project and its ultimate cancelation. Grassroots organizations, amalgamated under the banner ”Stop Spadina Save Our City Coordinating Committee” (SSSOCCC), functioned as a central opponent to the project (IC1. Toronto joined the opposition and was now actively fighting against the project. SSSOCCC’s primary goal to Stop Spadina Expressway is or-decomposed into their efforts to lobby Metro, the the Ontario Municipal Board (OMB), and the Province. By 1970, construction of the first segment (i.e., Wilson-401 to Lawrence, see Figure 7.2 for segment map) of the Spadina Expressway was complete, but with budget overruns. There was significant growth in the outer suburbs and infill in the city centre where the expressway was to be built, and attitudes about expressway developments and the importance of neighbourhoods changed. Metro required approval for additional loans for the expressway from the OMB (see Figure 7.5 for the additional decomposition from Get Funds From Metro). Metro and the OMB were still focused on development through infrastructure Chapter 7. Validating Evolving Intentions in Tropos 116 projects, and both ignored SSSOCCC’s lobbying efforts. However, SSSOCCC was successful in persuading the Province, which also controlled funding, that the original plan for the Spadina Expressway should not be built because it no longer served the people. In Figure 7.5, the --S relationship between Block Spadina and Approve Funds, within the actor boundary of the Province, indicates that when Block Spadina is Satisfied, then there exists full evidence that Approve Funds is Denied (as demonstrated in Figure 7.5). Since the Spadina Expressway was not built, we focused our investigation on how stakeholders gave closure to the project. Construction was finished by satisfying Build Lawrence to Eglinton, within the decade (IC3), and was funded (IC4) by Metro and the Province.

7.2.3 Scene III (1947–1953)

For Scene III (see MPLAN in Figure 7.6), we went back in time (prior to Scene I and Scene II), and focused on the intentions of the Toronto and York Planning Board (Planning Board). Planning Board was the initial supporter of the project (IC1) and were motivated by their desire to Have a Uniﬁed Arterial Road System. The Spadina Project was seen as one component of the Network Project aligning with Toronto’s ‘Master Plan’ to Be a World Class City and Grow City. Looking at the network of roads, there were two options: Build Road Network - Single Project, where Planning Board treated the whole road network as a single project; and Build Road Network - Multiple Projects where Planning Board treated each highway as a single project (IC2). Toronto and York were the possible funders of this network (IC4), with York actively opposed to both funding the project and the Planning Board’s activities. Toronto agreed to pay for two-thirds of the startup costs (i.e., satisfying Get Startup from Toronto), but York refused to pay the other one-third. Planning Board wanted the entire network to be built, but no concrete timelines for the expressways were determined (IC3) because Planning Board was unable to persuade York to fund the network. Planning Board strongly believed that York would not overturn their decision to deny the board funding (i.e., Get Startup from York has a Constant function with the value Denied, FD, in Figure 7.6), so they considered alternative means to acquire the funding. The most likely way to force York to fund the project was to amalgamate the region enabling Toronto representatives to outvote those from York. This is represented by the additional goal Amalgamate Toronto and York. This goal was achieved in 1953 with the inauguration of the Municipality of Metropolitan Toronto.

7.3 Simulating Scenes

In this section, we use the scenes introduced in Section 7.2 to generate simulations of the historical events and alternative futures (probed using the questions in Table 7.1) in order to provide evidence for research questions RQ 7.2 and RQ 7.3. We use strategy ASR2 (see Section 4.3.1) to generate Graph Evaluation Paths (see Definition 4.5) in Tropos (see Section 4.1.2 for relevant background). We prevent Strong Conflict, so that no intention has the evidence pair (F, F) meaning that it is both Fully Satisfied and Fully Denied.

7.3.1 Scene I (1960–1964)

MPRO Simulation (evidence for RQ 7.2). We created a simulation of MPRO , between Jan. 1960 and Jan. 1984, with time points (0, 3, 4, 5, 7, 8) (see Video I Hist online1). At the beginning of the Chapter 7. Validating Evolving Intentions in Tropos 117

Figure 7.6: MPLAN : Scene III fragment at t = 12 (July 1953).

simulation, Plan for Shopping Plaza is satisfied. As the simulation progresses, Metro satisfies Approve Project Funding as well as Approve Construction to Lawrence and Approve Plan in the Spadina Project, at t = 3 (July 1961). The Province then satisfies Approve Funds. These approvals are denoted by the FS label on each intention. Next, Build Mall has the value PD at t = 4 (Jan. 1962) and PS at t = 5 (July 1962). Figure 7.4 shows the results of the simulation at t = 4 (Jan. 1962). In subsequent time points, Build Wilson-401 Lawrence and Build Lawrence to Eglinton are no longer Denied (see Figure 7.2 for the project map). At the end of the simulation, t = 8 (Jan. 1964), Have Spadina Expressway is still Denied, and both Build Wilson-401 Lawrence and Build Lawrence to Eglinton are Partially Denied.

Alternative (evidence for RQ 7.3). Looking at the interactions between the Spadina Project and the Yorkdale Project, we considered the inevitability of satisfying Build Wilson-401 to Lawrence, and explored question AQ9 in Table 7.1: What if the Yorkdale project was not developed? For this alternative, we assigned Denied FD to each intention of the Yorkdale Project and given a Constant function. We changed the evolving functions for Build Wilson-401 to Lawrence and Build Lawrence to Eglinton, as well as other details available online. We generated multiple simulations and observed that without the Yorkdale Project, building the Wilson-401 to Lawrence segment may still have occurred but was not inevitable. Simulation results varied between completing no construction prior to maxTime to partially completing a segment but not necessarily Build Wilson-401 to Lawrence. This suggests that an alternative ordering for construction (e.g., satisfying Build Bloor-Sussex To Lakeshore ﬁrst) could have been possible without the inﬂuence of the Yorkdale Project. Chapter 7. Validating Evolving Intentions in Tropos 118

Table 7.1: List of analysis questions for the Spadina Expressway. Entries for each question list ‘++’, ‘+’, or ‘-’, specifying that the model can fully (++) or partially (+) answer the question, or indicate that the question is outside the scope of the model (-).

List of Questions MPRO MOPP MPLAN MFULL AQ1 Should the Spadina Expressway have been built? If so, + + + ++ how? AQ2 How can the highway development proceed as a single - - ++ ++ project (w.r.t. funding, construction)? AQ3 What if York satisfied Approve Planning Board Funding Y - - ++ ++ prior to 1950? AQ4 Can the Spadina Expressway be built without govern- + + + ++ ment/city funding? If so, how? AQ5 What if SSSOCCC had successfully organized in 1961? - - - ++ AQ6 What if SSSOCCC had never successfully organized? - ++ - ++ AQ7 What if all land was expropriated prior to development? + + - + AQ8 What if the Liberal party had won a provincial election ++ ++ - ++ during the project? AQ9 What if the Yorkdale project was not developed? ++ - - ++ AQ10 What if the Crosstown Expressway was not cancelled in ++ - - ++ 1961? AQ11 Is it possible to satisfy Have a Unified Arterial Road System - + - ++ by 1980? If so, how? AQ12 What if the downtown section (including Cedarvale) was + + - + built first?

7.3.2 Scene II (1970–1985)

MOPP Simulation (RQ 7.2). We created a simulation of the events between Jan. 1970 and July 1985 1 (maxTime = 31), using MOPP (see Figure 7.5). The visualization (captured online in Video II Hist) has time points (0, 2, 3, 4, 12, 13, 22, 30). At the beginning of the simulation, construction is underway on the Lawrence to Eglinton segment (see Figure 7.2). Metro’s task Consult with Citizens is Satisfied, but Listen to SSSOCCC is Denied. With the exception of Toronto who has joined the opposition, all of SSSOCCC’s intentions to lobby other actors remain Denied, until t = 3 when the Province Satisfies Listen to SSSOCCC and Block Spadina, which results in the Denial of Get Funding in the Spadina Project actor. Figure 7.5 depicts MOPP at t = 12, but all the changes that occurred at t = 3 are visible as they did not change afterwards. At t = 12 Toronto finished their task to Create Central Area Plan, which is adopted by Metro at t = 22 with the satisfaction of their goal Adopt Centres Policy. Eventually (t = 30), the Province satisfies their task Lease Toronto Blocking Land which prevents future development on the Spadina Expressway. Alternative (RQ 7.3). In the end, it was determined that the Spadina Expressway should not be built in some part due to the lobbying efforts by SSSOCCC. Next, we ask question AQ6 (see Table 7.1): What if SSSOCCC had never successfully organized? We assigned Denied FD to all of the lobbying efforts Chapter 7. Validating Evolving Intentions in Tropos 119 by SSSOCCC (other details are discussed online1) and created a new simulation that resulted in the completion of the Spadina Expressway (i.e., Have Spadina Expressway becoming Satisfied FS). This led us to realize that we had not modeled the impact of Toronto’s opposition to the Spadina Expressway and wondered if that opposition would have been sufficient to stop the expressway. See Section 7.5.1 for a discussion.

7.3.3 Scene III (1947–1953)

MPLAN Simulation (RQ 7.2). Using MPLAN (see Figure 7.6) as input, we generated a simulation of the events from July 1947 to July 1953 (maxTime = 12). The resulting simulation (see Video III Hist online1), with time points (0, 1, 3, 4, 9, 12), shows that very few goals are becoming Satisfied, and there is little movement on the Spadina Project or the Network Project. At t = 3, t = 5, and t = 9, changes occur in Get Crosstown Design Approved and Get Lakeshore Design Approved, but neither become satisfied. Planning Board believed that the project could not move forward without funding from York and that York would continue to refuse funding indefinitely. Planning Board had the task Lobby for Singular Municipality to remove the responsibility from York. This task becomes Satisfied at t = 12 (see also Figure 7.6).

Alternative (RQ 7.3). The goal Have a Unified Arterial Road System in Network Project (see Figure 7.6) can be satisfied by one of two tasks: Build Road Network - Single Project or Build Road Network - Multiple Projects. In the original simulation, Build Road Network - Multiple Projects was selected (but not Satisfied). We consider this trade-off as an alternative future, asking the question AQ2 (see Table 7.1): How can the highway development proceed as a single project (w.r.t. funding and construction)? Starting with the initial simulation, we assigned Build Road Network - Single Project a Stochastic-Constant (RC ) function (with FS as the constant value). RC specifies that the evidence pair assignment of an intention starts a stochastic function over one interval and then remains constant, in this case Satisfied, over a second interval. We also removed some absolute time assignments (i.e., constraints in MPLAN .MC). However, no simulation could be generated as the model was over constrained: Get Network Funding could not be satisfied without York’s approval of initial funding. We then considered a scenario where York had satisfied Approve Planning Board Funding Y prior to 1950 (AQ3), by changing the evolving function for Approve Planning Board Funding Y to be a Denied-Satisfied function transitioning in 1950. In the simulated timeline, this results in the satisfaction of Build Road Network - Single Project, but only if Get Road Funding is approved. Our model agrees with the stakeholders’ decision to build roads individually.

7.4 Spadina Expressway Model (1947–1985)

In this section, we focus on modeling and understanding the entire Spadina Expressway Case taken together. We construct the full model, using our merge algorithm, by combining the scene models together and adding missing information. We then explore the events of the full timeline and evaluate project trade-oﬀs through simulation.

7.4.1 Merge Algorithm

In this section, we describe the merge algorithm and demonstrate it on MPRO and MOPP . In this study we used this algorithm to merge MPRO and MOPP and repeated the process to merge the result with MPLAN , in order to create MFULL in Figure 7.9. Chapter 7. Validating Evolving Intentions in Tropos 120

Figure 7.7: Illustration of merge of MPRO and MOPP .

Merging is a semi-automated approach to unify information from two evolving goal models, speciﬁed over two absolute time periods. Inputs to merge are two evolving goal models M1, M2 = hA, G, R, EF , MC, maxTimei. The user must manually specify absolute start and stop time points for each model based on the real-world values for maxTime, such that M1.start < M2.start and M1.stop < M2.stop. For example, Figure 7.3 illustrates the real world relationship between MPRO and MOPP . Using the mapping, we set MPRO .start = 0, MPRO .stop = 8, MOPP .start = 20, and MOPP .stop = 51. Output of the merge process is a merged model (Mm), such as MIM – the result of merging MPRO and MOPP (see Figure 7.3). The steps are as follows:

(1) Create the timeline for the merged model. Mm.start = M1.start, Mm.stop = M2.stop. E.g.,

MIM .start = 0 and MIM .stop = 51. (2) Update symbolic and absolute constants in each model (i.e., MC, EF , and R). Prior to merging, all absolute assignments were specified relative to 0 and maxTime (see Section 7.2.1). First, we verify whether the naming of symbolic constants between the two models is unique and prompt the user to verify non-unique names. Second, we update any absolute time assignments to make them relative to M1.start. Third, we update references to the values of 0 and maxTime in each model to that model’s start and stop times. For example, Build Lawrence to Eglinton in MOPP was assigned a UD function over two intervals, with a transition between them occurring at time 13. It is now updated to 13 + MOPP .start = 33. (3) Union merge actors and intentions (i.e., A, G) based on element names [109]. We require a unique identification of intentions. Conflicts may arise when performing a union of the intentions if two intentions have the same name but are not elements of the same actor. In this case, we prompt the user to resolve the conflict. For example, Figure 7.7 shows snippets of the MPRO model from Figure 7.4 and the MOPP model from Figure 7.5. In this step, any actor and intention that appears in either model will appear in the merged model (see Merge of MPRO & MOPP in Figure 7.7). No links are yet created. (4) Merge relationships in the model (i.e., R). We automatically add identical relationships and relationships whose goals only exist in one model, and prompt the user to resolve the remaining relationships. For example, if two goals are connected with a ++ relationship in one model but are not related in the second, the user must resolve the conflict. For example, when merging the relationships in Figure 7.7, the ++ link between Approve Project Funding and Get Funding From Metro was present in MPRO but absent in MOPP ; yet both tasks were present. In this case, we did not put the link in the merged model because it was refined in MOPP . Chapter 7. Validating Evolving Intentions in Tropos 121

0 78 20 33 51 Scenes I & II (⊥, F) (⊥, P) (P, ⊥) (F, ⊥) Merge of I & II 0 7 12 33 51

Figure 7.8: Resolving MPRO and MOPP merge of evolving function for Build Lawrence to Eglinton.

(5) Prompt the user to define “presence conditions” over the model. Update evolving functions for regions with presence conditions. We use the term presence condition to indicate that a particular part of the model exists only for a particular duration of time. For example, in MOPP , the intentions within the boundary of Toronto were only present after and including July 1961, so we define Toronto’s presence condition as [3, 51], meaning that Toronto is present between t = 3 and t = 51. All intentions within Toronto are assigned the special value Nonexistent, NE, for the period [0, 3). This process is discussed further in Section 7.4.2. (6) Merge evolving functions (i.e., EF ). For all intentions that exist in only one model, we prompt the user to either review each function to add further specification, or leave all functions outside the previously defined time period undefined. For all intentions that exist in both models, we give the user one of three options: (a) If the models align (M1.stop = M2.start), we prompt the user to either automatically connect each function, or review each function. (b) If the models cover non-consecutive periods

(M1.stop < M2.start), we prompt the user to either review each function to add further specification, or leave the intermediate time period undefined. (c) If the models overlap in time (M1.stop > M2.start), we prompt the user to either automatically connect each function giving priority to one model in the overlapping period, or review and assign each evolving function and value manually. For example, since MPRO .stop = 8 and MOPP .start = 20, Option (b) for merging MPRO with MOPP applies, so we reviewed and updated each function. Build Lawrence to Eglinton was defined in both models with a UD function, as illustrated in Figure 7.8. The period between 8 and 20 is not yet specified, and we could have left the function undefined; instead, we connected the two functions, transitioning from (⊥, P) to (P, ⊥) at t = 12. The above algorithm is a two-way merge (i.e., it does not rely on a common ancestor model) [106]. The success of merge rests on a single modeler creating all models, which results in four assumptions: • Models have common naming and terminology for intentions and actors. • Models are at the same level of abstraction and have the same boundaries. • Modelers have a similar fundamental understanding of the use of notation. • Models have a single view of evolving functions and relationships. Given these assumptions, we do not detect conflicts or inconsistencies between the models, with the exception of overlapping time periods in evolving functions for intentions and relationships (as described above). In order to merge models created by different modelers we would need to be able to detect these conflicts and inconsistencies. We can do so by constructing a three-way merge [106], where there is a third model that acts as a common ancestor giving a mapping between the two original models. Prior work on three-way merging would need to be extended to address conflicts within evolving intentions. In Section 8.2, we further describe possible improvements to the merge algorithm. Chapter 7. Validating Evolving Intentions in Tropos 122

7.4.2 MFULL: The Complete Model

Figure 7.9 shows MFULL, the complete model for the Spadina Expressway project. The model contains the ten actors studied in this case: Spadina Project, Yorkdale Project, Network Project, Toronto, York, Metro, OMB, SSSOCCC, Province, and Planning Board. Between them, these actors have 124 intentions and 154 relationships.

Actor Presence. As mentioned in Step (5) of the merge algorithm (see Section 7.4.1), the user assigns presence conditions in the merge process, visualized by drawing bubbles (i.e., loops) around regions of the model, and assigns presence conditions to each bubble, which are then assigned to each intention within the bubble. The model in Figure 7.9 is annotated with the bubbles and presence conditions that we assigned while creating the full model of the Spadina Expressway project. For example, Toronto has two bubbles: one for when it supported the Spadina project ([0, 27]) and another for when it opposed it ([28, 76]).

7.4.3 Analyzing the Complete Model

MFULL Simulation (answering RQ 7.2). With MFULL, we generated a simulation of the complete historical timeline for the Spadina Expressway project (see Video X Hist online1). The simulation has the time points from each of the scenes as well as the changes that occurred between scenes: (0, 1, 3, 5, 7, 9, 12, 13, 14, 15, 16, 21, 23, 25, 27, 28, 29, 30, 31, 32, 33, 34, 37, 38, 45, 47, 48, 49, 57, 58, 61, 64, 67, 75). In summary, the evolving intentions framework with path-based analysis is able to fully reproduce the timeline of actual events as documented based on our rational reconstruction (for both the scene models and MFULL), addressing RQ 7.2. Alternative (answering RQ 7.3). Having reproduced the timeline of historical events, we can explore alternative scenarios across the whole project. Consider AQ11 (see Table 7.1): Is it possible to satisfy Have a Unified Arterial Road System by 1980? We start by removing the evolving functions for each of the building tasks (e.g., Build Bloor To Lakeshore) and allow the analysis to choose values for these tasks. Other changes are documented online1. With these new inputs, we created a simulation result where the expressways were built by 1980. The simulation did not assume that the Spadina Expressway would be build from North to South, and instead selected segments at random. When all segments of the Spadina Expressway were build prior to 1971, then Have Spadina Expressway and Have a Unified Arterial Road System were temporarily Satisfied (at least until Province removed funding). In summary, using the Evolving Intentions framework with path-based analysis, we were able to produce alternative futures, successfully answering RQ 7.3. Furthermore, by generating alternative scenarios, we discovered and corrected incompletenesses in the original model improving the overall quality of the model.

7.5 Results and Discussion

In this section, we answer RQ 7.1, and discuss observations made during the study and their implications on future research. We ﬁnish by discussing threats to validity and answering our overall research question. Chapter 7. Validating Evolving Intentions in Tropos 123 [0-12] [26-76] [18-76] [13-76] [24-76] [13-76] : Spadina Expressway project model (1947–1985). [13-76] [0-12] FULL M [28-76] Figure 7.9: [0-27] [0-12] Chapter 7. Validating Evolving Intentions in Tropos 124

7.5.1 Modeling & Analysis

Challenges Gathering Data. Not all the required information was available in the historical documents. For example, it was challenging to decide the evolving intention for the task Get Crosstown Designed Approved because the historical documents stated that the Crosstown was removed in 1955 and 1961, and it was unclear when in the interim the Crosstown was added back into the design. We experimented with two different functions and determined that they did not impact the model overall, so this ambiguity was accepted. It was sometimes difficult to determine the evolving functions of some of the intentions from the historical documents, because they were written for a different purpose. The original source material used by the authors of the books we read (e.g., meeting minutes) did not prove helpful because they lacked contextualization. We believe that this is not a weakness of the Evolving Intentions approach but rather the fact that we did rational reconstruction. For example, we had hoped to capture the intended timelines for the construction at specific time points; however, the historical documents only contained this information in generalities, so we used estimates to generate alternative timelines.

Modeling Challenges. As mentioned in Section 7.4.3, we had difficulty modeling “continuous” funding behaviour. For example, in creating alternative futures, we defined a scenario where both Metro and Province funded and built the Spadina expressway resulting in the satisfaction of Have Spadina Expressway. After Build Spadina Expressway was satisfied, Province would no longer need to fund the expressway, and funding levels could become Denied; but this would result in the denial of Have Spadina Expressway due to the nature of the And-Decomposition. Goal models were not originally intended to model behavioural aspects; instead, we modeled funding projects as tasks that are completed, as opposed to continuous levels as in stock and flow diagrams [138]. The framework does not allow for changes in the names of actors and intentions. In the historical documents, York could refer to Township of York, Borough of York, or the City of York, none of which should be confused with the Regional Municipality of York. Toronto might also refer to Old Toronto, the City of Toronto or a particular sub-committee of the City Council. We were unable to visualize such name changes and thus considered them all as the same entity. Much of the discourse among proponents and opponents of the project was about convincing others to change their opinion and/or shift their priorities. It was difficult to represent power struggles in the model. For example, we modeled the influence of SSSOCCC on other actors through lobbying tasks and the effectiveness of lobbying as tasks labeled Listen to SSSOCCC. This limitation comes from the underlying GORE approach [69].

Externalities of Analysis. Creating alternative futures allowed us to understand limitations of our initial models. For example, in Scene II, as we modeled the project with the activities of SSSOCCC, we realized that we had not modeled the impact of Toronto’s opposition to the Spadina Expressway, resulting in us adding the additional task Listen to Toronto in the actor boundary for the Province, and connecting this task with both the Province’s Block Spadina task and Toronto’s Oppose Spadina Expressway task. Our models did not capture all possible constraints; and therefore, simulation may produce unrealistic scenarios. For example, one of our simulations resulted in the satisfaction of Have a Uniﬁed Arterial Road System in three years. This result is unrealistic given our real-world knowledge of how long it takes to build a highway. In this case, the model was missing timing constraints on required time for road construction. Chapter 7. Validating Evolving Intentions in Tropos 125

As with all automated analysis techniques in GORE, results should be reviewed by stakeholders.

7.5.2 Discussion of Results

Answering RQ 7.1. Given the challenges discussed in the previous subsection, we return to RQ 7.1: To what extent can we capture changes in the elements in our universe (defined by IC1–IC4) with the framework? We conclude that the Evolving Intentions framework was successful at capturing all changes required to understand the case and answer questions. The framework could be improved to allow for changes in the names of actors and intentions. Merge. In the absence of tool support for auto-merge, we performed the merge manually (see Sec- tion 7.4.1) by adding the scenarios to a single canvas in BloomingLeaf, and then matching and merging information about each element using our expert knowledge of the semantics of the model. In some cases, the model was simplified by redrawing individual elements rather than merging multiple ones. In merging the Scene models, we encountered the trade-off in how to model Toronto’s intentions before and after 1961. As a supporter of the project in Scene III and an opponent in Scene II, Toronto’s goals were in conflict. We first tried modeling Toronto as two different actors and then merged their intentions into one actor afterwards. At the time of writing, BloomingLeaf did not distinguish between active and inactive versions of actors and was unable to visualize Nonexistent intentions. Our workaround was to create space in the middle of Toronto to make this distinction. Future research is required to validate the merge algorithm and implement tool support for auto-merge. Model Creation Process. In this chapter, we demonstrated our process for creating evolving models based on merging scenes that focus on groups of actors (see Section 7.4.1), with the goal of answering time-based questions. Table 7.1 lists each question and whether it can be answered using each model. While some questions could not be fully answered by the scene models (e.g., AQ5), it was more convenient to use these models to answer questions about a specific model part or time period (e.g., AQ9). While we found this process useful, this study does not validate its effectiveness. Further research is required to evaluate this process and compare it to generating a single large model and using slicing to answer specific questions.

7.5.3 Threats to Validity

We use a case study-speciﬁc classiﬁcation of threats to validity [126]; however, given that our rational reconstruction did not involve interviewed or observed persons, we had to adapt it. Construct Validity. Given that we conducted this study using our own framework, there is no risk of misinterpreting the framework’s constructs. There is a risk that we misinterpreted the meaning of constructs in the historical documents. We mitigated against this threat by providing traceability between model elements and the historical documents as well as conducting this study over the period of months and returning to the historical documents several times. Internal Validity. We do not attempt to make any causal claims in this work, but showcase how our technique can be applied to a large real-world example. As mentioned above, there is a risk that there may be errors in our interpretation of the historical documents and, by extension, our modeling and simulation results. These errors would threaten the interval validity of the results of the rationale reconstruction process but not necessarily signal a problem with the framework itself. Chapter 7. Validating Evolving Intentions in Tropos 126

External Validity. This study gives evidence to the generalizability of the evolving intentions framework for analyzing historical case studies. There is still a notable risk that this approach is not gener- alizable to allow stakeholders to make predictions about projects in the future. As this was outside the scope of the design of this study, further evaluation of the Evolving Intentions framework is required. Since the domain under study was outside software engineering, there is a risk that these results may not generalize to software projects. Reliability. All case studies involving goal modeling have a reliability risk, and our study is no exception. This is due to the open-world nature of goal modeling and the degree to which model generation is dependent on the subtle decisions made by the modeler. Given that individuals interpret data diﬀerently, reproductions of this study using the same historical documents will result in diﬀerent models; however, we believe that the process can be reliably used by other researchers. This study triangulates the results of our prior controlled experiment [61].

7.5.4 Answering the Overall Research Question

We modeled the Spadina Expressway project by creating smaller scenes and then produced the full model. We were able to investigate RQ 7.1–RQ 7.3 in each of the three scenes and the full model. By simulating the historical timelines and alternative futures, we demonstrated the analysis of this approach. This allows us to answer the overarching research question (RQ 7) aﬃrmatively: this rationale reconstruction using the Evolving Intentions framework was successful in capturing the evolution of the Spadina Expressway project.

7.6 Summary and Related Work

In this chapter, we validated the applicability and effectiveness of the Evolving Intentions framework in Tropos. In the absence of ground truth about future evolutions, we used historical data and rational reconstruction to understand how a project evolved in the past. We selected requirements of the Toronto Spadina Expressway, because it spanned a substantial period of time, had varying stakeholders, and was well-documented. In this chapter, we reported on our experience with rational reconstruction and the results of modeling the Spadina Expressway project over three different time periods. We demonstrated a novel process for creating time-based models through the construction and merging of scenarios, which resulted in createing a model representative of the complete timeline. We found the analysis capabilities to be sufficient for generating alternative futures. We conclude that the Evolving Intentions framework has sufficient expressive power to capture the evolution of the Spadina Expressway project. Changes in Actors. In this chapter, we looked at modeling the addition and removal of actors. TimedURN [10–12,99] supports the addition and removal of actors through DeactivationChanges, where an actor is still part of the model but is greyed out. We aim to provide a similar visualization in our simulations. TimedURN captures evolution of a goal model using a 150% representation. On the surface, our merged model (MFULL) has many similar elements (i.e., bubbles discussed in Section 7.4.2); however, during the merge process, the user updates the evolving functions for each intention, removing traceability to the original scene models. The analysis capabilities of TimedURN differ from our work, in that we can prove reachability or unreachability of a particular future state (e.g., answering AQ11 in Table 7.1), whereas TimedURN can only check whether this state happens to be on the generated Chapter 7. Validating Evolving Intentions in Tropos 127 path. Methods. We built on the methodology of similar studies in RE, and followed the guidance provided by [126, 145]. Aprajita [10] validated TimedGRL with a reconstruction of reports from the Auditor General of Ontario. Letier presented reconstructions of the London Ambulance Service and San Francisco Bay Area Rapid Transit systems using SRS documents [90]. Our work is similar to these in that each uses third-degree data to validate their respective frameworks. As mentioned earlier, we considered using case reports from software engineering (e.g., [17,85,116]) but without documented future evolutions we were unable to establish ground truth. Part IV

Conclusions & Future Work

128 Chapter 8

Conclusions & Future Work

This chapter concludes the thesis. Section 8.1 recaps the contributions and highlights of the thesis, and Section 8.2 discusses limitations and improvements. Section 8.3 provides an overview of future research in understanding change and Section 8.4 presents our proposal to study the utility of goal modeling.

8.1 Summary of Contributions

In this dissertation, we aimed to improve the state of the art of goal-oriented requirements engineering; specifically, by looking at improving goal model analysis (i.e., making tradeoff decisions and asking “what if” questions) in the early-phases of requirements engineering. We observed that prior tradeoff analysis research may not result in the best decision if the fulfillment of intentions in the model varies over time and stakeholders cannot make guarantees about future evaluations. In this dissertation, we proposed and answered our central research question: how can we effectively model, reason with and support users in making tradeoff decisions given evolutionary information (RQ 1). We presented the Evolving Intentions framework for specifying, modeling, and reasoning about goals that change over time, enabling stakeholders to reason about model evolution. We examined Evolving Intentions in the context of both the iStar and Tropos goal modeling languages. We structured our work into three parts: the background and motivation for our work (Chapter1– 2), a complete presentation of the framework (Chapter3–4), and the implementation and validation of the framework (Chapter5–7). Throughout the dissertation, we used two motivating examples to give context to concepts. The Waste Management example (WME) looked at a city considering the tradeoff of building a new landfill as well as a recycling and composting facility. The WME was used to showcase relevant concepts in iStar. The Bike Lanes example (BLE) looked at a project to add bike lanes to a major residential and commercial thoroughfare in a large city. We not only used this example to motivate our work in Tropos, but also used a fragment from the example to introduce each concept in the Evolving Intentions framework in Chapter3 and Chapter4. We began, in Chapter2, by introducing relevant terminology and examples throughout the thesis. We introduced the syntax and semantics of a goal model (or goal graph) in both iStar and Tropos. First with iStar, we focused on the visual syntax and showed how evaluation labels can be propagated through forward and backward analysis. Since we primarily extended Tropos in this dissertation, we gave a complete presentation of the formal syntax and semantics of Tropos. We presented algorithms

129 Chapter 8. Conclusions & Future Work 130 for propagation of evidence pairs as well as evaluating a goal graph as a whole, and illustrated these algorithms and other concepts with the Bike Lanes example. In Chapter3, we presented the modeling aspects of the Evolving Intentions framework. We demonstrated how stakeholders can model intentions with evolution, and ultimately gave a definition for an Evolving Goal Graph in Tropos. We first established our representation for time points, and then defined four atomic functions, with which stakeholders can specify evolving functions over disjoint neighbouring intervals (including repeating functions). Second, we showed how relationships change over time and created a notion for mutually exclusive decisions. Finally, we specified how users can create constraints between time points and illustrated the complete specification of the evolving goal graph for the Bike Lanes example. For each of these, we specified the notation, visual representation, and usage. In Chap- ter3, we also demonstrated how the Evolving Intentions framework would be implemented in iStar and how this differs from the framework definitions in Tropos. We also illustrated how stakeholders might specify changes in the Waste Management example (in iStar) with the Evolving Intentions framework. In Chapter4, we enabled stakeholders to answer questions with model evolution over time and described how to reason with Evolving Intentions. We first described, in Tropos, how goal graphs are evaluated at specific time points, which defined our notion of a state. We then composed these evaluations to create a simulation path, and specified a brute-force algorithm for finding graph evaluation paths, and illustrated the algorithm using the Bike Lanes example. We proved correctness and termination of our path algorithm. We then commented on how the reasoning would apply for iStar and detailed how iStar’s informal propagation rules limited interpretation of results. This limitation lead us to present a constant time algorithm in iStar for finding graph evaluation paths when evolution is restricted to leaf-level intentions. To connect simulation algorithms with stakeholder questions we specified four simulation strategies, and demonstrated how they can be used to answer the questions in the Bike Lanes example and Waste Management example. In Chapter5, we described the implementation of the Evolving Intentions framework. Since our brute- force algorithm introduced in Chapter4 resulted in an exponential runtime in the worst case, we encoded the problem of solving a graph evaluation path as a constraint satisfaction problem. In Chapter5, we briefly introduced this encoding and described our implementation of our tools: GrowingLeaf and BloomingLeaf. These web-based modeling and analysis tools implemented components of the Evolving Intentions framework in iStar and Tropos, respectively. For each tool, we detailed the features and architecture and how users can answer questions about the evolution of a goal model. In Chapter6, we validated the Evolving Intentions framework in iStar by studying the effectiveness, scalability, and usability of the framework. We initially demonstrated that the Evolving Intentions framework was effective for debugging goal models and answering stakeholder questions using the development of GrowingLeaf as an example. We measured the scalability of our approach, by creating four tests that varied the size of the graph and the number of time points in a path for each simulation strategy. We then used two example models and a series of randomly generated models to show the analysis to be scalable for these models. We investigated usability by completing a between-subjects controlled experiment with graduate students. In Section 6.2, we described this experiment in detail and showed how the results empirically validated the effectiveness of our approach and tool usability. In Chapter7, we validated the Evolving Intentions framework in Tropos. We reported on the applicability and effectiveness of our technique on a substantial case. We presented a rational reconstruction of the Spadina Expressway project, by building three scene models from historical data and merging them Chapter 8. Conclusions & Future Work 131 to create a model representative of the complete timeline. We demonstrated the analysis capabilities by illustrating how the project evolved in the past and generating alternative futures to make predictions. We concluded that the Evolving Intentions framework has sufficient expressive power to capture the evolution of the Spadina Expressway project.

8.2 Limitations and Improvements

Here we describe future work that can mitigate the limitations and improve the contributions of this thesis. Approach Improvements. In this thesis, we used two separate motivating examples to introduce the framework in iStar and Tropos. We also did not compare the use of the framework among users of iStar and Tropos, or evaluate the languages side-by-side with users or a single example. We want to determine whether the change in syntax and the addition of the ability to specify absolute time results in an observable improvement (or decline) in two areas: (1) quality and expressiveness of stakeholders’ models and (2) the questions that can be answered using the analysis capability of the tool. Ultimately, we will explore whether the additional functionality and different syntax enable the users to make better decisions. We will also explore overlaying symbolic satisfaction values (e.g., and ) on top of evidence pairs in Tropos. Depending on these results, we may make the framework and tool compliant with the new iStar 2.0 Language Guide [35]. In Section 3.1.5, we listed the limitations of the Evolving Intentions framework. Future work will expand the expressive power of the formalism. In Section 4.3.1, we defined strategies to connect analysis procedures to stakeholder questions, but we still do not know if stakeholders can make the connection between their questions and our strategies in order to select an appropriate analysis technique. Further research will explore connecting stakeholder questions with analysis strategies and methods. In Chapter7, we introduced an algorithm for merging models at different time points with Evolv- ing Intentions. Future work will formally validate our merge algorithm and consider which properties are maintained. Furthermore, as discussed in Section 7.4.1, the work in this dissertation assumes the perspective of a single modeler, and we did not investigate merging evolving model elements between modelers. Future work will reconsider our merge algorithm as a three-way merge [106], and investigate how to resolve conflict in evolving intentions as well as merging multiple view points [129] in the context of the Evolving Intentions framework. While developing the Evolving Intentions framework, collaborators suggested that the framework could be useful for identifying and analyzing project obstacles and hazards, such as in the work of Letier [90]. Future work will create a methodology for obstacle analysis in the context of evolution. We could also look at how goals and obstacles affect risk management more broadly as in the work of Islam and Houmb [79]. Furthermore, we could develop a methodology to connect our approach with scenario models to help stakeholders discover goals and evolution in their domains (see Section 4.4 for further discussion). In Section 1.2, we stated that we chose iStar and Tropos so that we could help stakeholders collaborate and understand each other’s intentions in making trade-off decisions in the context of evolution. This dissertation is therefore limited because due to external constraints, we have yet to validate the framework with multiple stakeholders. Furthermore, we did not formally define the concept of actor availability in the Evolving Intentions framework. In Section 7.4.2, we describe using presence conditions for actors as Chapter 8. Conclusions & Future Work 132 part of our model merge algorithm. Future work will improve the framework to consider the evolution of actors. Finally, to fully support dialog between stakeholders, we can enable actor-centric reasoning, where analysis methods either prioritize or isolate a subset of actors in the model and consider only their intentions and dependencies. We hope this will improve the versatility of our analysis, and mitigate lengthy analysis run-times over larger models, when not all actors are relevant to the query. In KAOS, an assumption is a statement to be satisfied by the environment [140]. These are explicit assumptions made by the system. In GRL, a belief is a fact that is important to a stakeholder [6]. Since we aim to deal with the intentions of multiple stakeholders, we must look broadly at assumptions and beliefs. In validating the Evolving Intentions framework we observed that stakeholders have tacit assumptions throughout the modeling process resulting in assumptions in their models. Stakeholders make assumptions about the domain and intended users of the system, as well as the intentions of other stakeholders. With respect to reasoning, modelers make assumptions about why the evolving functions and evaluations they selected are appropriate. Future work will explore and classify the different types of assumptions made both in the model and modeling/reasoning process. We will develop analysis techniques that will expose and challenge the implicit assumptions of stakeholders. For example, we can connect scenario models with the Evolving Intentions framework as in Rolland et al. [123] to generate additional goals and expose assumptions (see Section 4.4). We will also consider documenting these assumptions and representing them in the model, creating traceability links for stakeholders. Validation Improvements. Considering the limitations and threats to validity already discussed in Section 6.2.3, future work will focus on improving the controlled experiment, generalizing our findings to a broader population (including industry-based modelers). Future work will also replicate these results with other subjects and domains, and evaluate other analysis techniques within our tool. In Chapter7, we showed how the framework can explore questions using a historical case study. As discussed in Section 7.5, we hope to further validate the Evolving Intentions framework with an external industrial case study from software engineering. We would like to evaluate the Evolving Intentions framework with a longitudinal study where stakeholders might actually observe each other’s intentions and their models evolving. This dissertation is limited by the single software example in Section 6.1.1. We need to further validate the framework with examples from software engineering. In addition to modeling project evolution, we anticipate using the Evolving Intentions framework to model trade-offs in ethical decisions. For example, in the context of autonomous vehicles, we could explore implications for the collision avoidance system in choosing between harming occupants of the autonomous vehicle, occupants of other vehicles, or unprotected road users (e.g., cyclists and pedestrians). These ethical decisions affect non-users of autonomous systems and their designs must broadly consider the intentions of all stakeholders and society as a whole. Tooling Improvements. At the time of this writing, our implementation is limited to finding full paths. We are currently implementing an all paths analysis feature in BloomingLeaf. This would give users the ability to explore all possible next states of a model and create their own simulation paths. Future work is required to add model management features as well as additional syntax checking. For example, we will give users the ability to save and load analysis configurations and results. We also intend to implement the semi-automated merge algorithm presented in Section 7.4.1. We will explore adding visualizations for analysis trends, similar to those in TimedURN [99]. Further work is necessary to evaluate the usability of our interface and the effectiveness of our tool with user studies. Chapter 8. Conclusions & Future Work 133

8.3 Future Work: Extended Support for Understanding Change

Projects are more likely to be successful if there is a way to anticipate and address change. In this thesis, we created a framework for modeling and reasoning with known changes in early-stage goal- oriented requirements engineering. Our goal was to support decision making in the presence of change, across a variety of situations. In the future, we will focus on change management, support for multiple modelers, and exploring other domains.

Change Management. Not all changes are known in the early-stages of a project; thus, stakeholders need to be able to update models and deal with unanticipated change. A fundamental assumption in our current work is that stakeholders can create a complete representation of their model, knowing all goals and all possible changes at the start. An important insight from the autonomous systems community is to separate the specification of a change from the system behaviour [23]. Although goal models do not deal with behavioural aspects of systems, we intend to leverage this insight and critically evaluate how we conflate the specification of a change with the context of that change. We believe that making this separation explicit will result in a suitable representation for handling unknown changes and adapting the model over time. This will enable the community to connect models with downstream activities, informing runtime scenarios.

Support for Multiple Modelers. All interesting systems are developed by teams. Current tooling and analysis is predicated upon a single user completing tasks. Having multiple stakeholders create and merge models is interesting not only from a technical stance but also in terms of group dynamics. We will create and assess heuristics for unifying inconsistent evolutions and will also explore conﬂict resolution and psychological safety [40] in the context of early-stage requirements modeling. This research can be done both in the classroom and in an industrial context. With an industrial focus, we can address and eliminate barriers to the adoption of our technique, and use industrial models as further validation. With an education focus, we want to improve how we teach interactions in group projects (to help students create better teams), and understand if psychological safety can be achieved during the short timeline of a course project. Working with multiple users will not only improve the adoptability of our approach but the lessons learned will likely impact other approaches to requirements engineering.

Exploring Application Domains. The techniques that we have developed thus far have applications in other domains. For example, we expect to apply our analysis techniques to the problem of managing uncertainty in software design [42]. Specifically, our work is directly applicable to the planning of design uncertainty resolution. We intend to create techniques for the analysis and visualization of time-based queries over software with design uncertainty, and empirically study the effectiveness of these extended approaches with users. Safety standards are created to ensure the safety and security of customers. In order for manufacturers to become certified as complying with a standard, they must create a set of assurance cases giving evidence and arguments that the required safety goals are met. There exists an overlap between goal models and certification [130]. Although the syntax of goal models does not have a notion of an evidence- based argument, we can model the evolution of an assurance case, and ask questions about the past, present, and future compliance of our system given changes to the specification. By studying these domains and their users, we will not only explore the broader applicability of our framework, but also hope to understand patterns that result in change and devise methods to anticipate change. Chapter 8. Conclusions & Future Work 134

8.4 Future Work: Utility of Goal Modeling

In this dissertation, we provided qualitative analysis through simulation allowing stakeholders to consider alternatives over time and answer time-based questions using goal models. We used automated analysis and enables stakeholders to visualize single simulations and predict outcomes [61]. Although goal modeling has long been used in the literature, examples of its adoption in industry are limited. Others have surveyed the barriers to adoption of software engineering research more generally. Lo et. al. found evidence that practitioners did not adopt SE research because research tools were not directly applicable to work tasks and the benefits did not outweigh the costs [96]. Ongoing work by Franch et al. is looking at how practitioners perceive RE research [45]. We hope their findings, once published, will document additional opportunities for encouraging GORE adoption. Mavin et al. surveyed experienced requirements engineering practitioners to investigate the lack of adoption of goal models [104]. They found that significant barriers to adoption among practitioners included a lack of tooling, expertise, and organizational enthusiasm, as well as a belief that the approaches do not scale or have sufficient benefits. In their survey of the literature they found that the vast majority of the validation in goal modeling papers focused on feasibility, with very few papers discussing practicality or utility. Furthermore, of the few controlled or quasi-experiments published in the goal modeling community, most have completely drawn models (e.g., [61], [132]) or starter models (e.g., [72]). Rather than devote significant effort to developing a professional tool, we assert that we must understand where to focus development efforts to mitigate these barriers and encourage adoption. In this work, we consider activities in GORE and question the assumptions that the approach makes, in order to identify the activities that do and do not benefit stakeholders and give recommendations for areas where automation would help increase adoption.

Activities and Assumptions. We again consider the Bike Lanes example to explore assumptions in our approach and goal modeling approaches generally. With a goal-oriented approach, (A1) we assume that modelers can construct a goal graph (or model) as shown in Figure 1.2. Recall from the partial goal model of the project (see Figure 1.2), the stakeholders want to satisfy Have Bike Lanes while satisfying Cyclist Safety and Access to Parking (i.e., some of the root-nodes in the graph). Next, (A2) goal-oriented approaches assume that modelers can use analysis to answer trade-off questions. An example “what if” question that stakeholders can answer using goal model analysis is, “If bike lanes are permanently installed, will Have Bike Lanes, Cyclist Safety, and Access to Parking be satisfied?” Finally, (A3) time-based goal modeling approaches assume the evaluations of some elements in a goal model change over time and that goal modeling can be used to express these changes and answer time-based questions. For example, Have Design became more satisfied at some point, but Bike Lane Usage varied depending on external factors, such as seasonal changes in weather. Permanent Construction affected Access to Parking while the roadway is being reconfigured, whereas Temporary Construction did not have the same effect. The city planners wanted to understand how changes in these goals impact their decisions by exploring WME-Q1–WME-Q6. Both TimedURN and Evolving Intentions argue that it is feasible for stakeholders to create visualizations answering these questions. While GORE methods are successful in a number of ways, these assumptions (A1–3) have not been sufficiently addressed and leave some questions unanswered. For example, what benefit is gained from drawing the initial goal model in Figure 1.2? How do stakeholders know their model is valid? How do stakeholders know what analysis method to choose? Do stakeholders use the analysis results to justify Chapter 8. Conclusions & Future Work 135 their preferred answer or discover the best approach? How can stakeholders verify results of time-based analysis to ultimately find out if results were meaningful? In this work we reflect on some of these assumptions, and ask if they are valid or what impact they have on the utility of the approach overall. We suspect that these assumptions may underlie the limited adoption of GORE approaches discussed above. Contributions. We propose a protocol to investigate the utility of activities in GORE. We look at the process of goal modelling through the division of construction, analysis, and evolution. For each activity, we ask two research questions with the goal of understanding where the essential human practises are in GORE: To what extend do stakeholders gain utility in completing the activity? If there is limited utility to be gained, can simplification or automation of the activity provide similar utility with reduced effort?

8.4.1 Activities & Assumptions

In this section, we introduce assumptions made in GORE activities (i.e., construction, analysis, and evolution) and present potential research questions to examine assumptions. Our primary goal is to measure the ‘utility’ of each stage of goal modeling. Utility is either viewed as ‘fitness for some desirable purpose or valuable end’ or an ‘intrinsic property of anything that leads an individual to choose it rather than something else’ [112]. In this work, we are interested in the intrinsic utility of GORE rather than utility over other RE approaches (e.g., User Stories). We aim to compare the utility of completing activities in different ways. We begin by considering where in the GORE approach (i.e., in which activities) is the central utility. Much of the work on analysis and extensions of goal modeling languages (e.g., extensions for security, preferences, legal, regulations, adaptions, and evolution [2]) takes the construction of models as a default, and makes the assumption that the majority of the utility in GORE approaches is in analysis activities. However, in Horkoff and Yu’s comparison of forward analysis techniques (i.e., i*, GRL, and Tropos), they found that each technique resulted in a different answer and recommended that the analysis techniques be used as heuristics for model exploration and debugging rather than decision making [77]. Understanding the utility of construction and analysis will be helpful for GORE adoption, and is also required in order to make any claim about the utility of the evolution activities. Construction. We pose the following research questions in order to motivate our work and explore assumptions in construction:

• To what extent do participants gain value in manually drawing goal models (in a tool or on paper)? • Can participants gain similar benefits from reviewing and correcting a model? • What benefits do visual representations have over textual representations? To what extent is a visual representation beneficial? • What other requirements practices are complementary to goal modeling or have similar benefits? • To what extent is making changes or correcting a model sufficient to understanding them? • How do these benefits change when there is a single stakeholder vs. groups of stakeholders?

In the motivating example, we can ask: What beneﬁt is gained from drawing the initial goal model in Figure 1.2? While this is not an exhaustive list of research questions, we use it to generate two hypotheses: (HC-1) When constructing goal models, there is no utility in drawing them on paper. (HC-2) Chapter 8. Conclusions & Future Work 136

When constructing goal models, there is no utility in drawing them in a tool. Through experimentation, we could reject both HC-1 and HC-2 and discover utility in both approaches. If we fail to reject both HC-1 and HC-2 then the central utility in GORE may be in analysis and there may be opportunities for automation of construction activities. Also, if we reject HC-1, fail to reject HC-2, and find utility in analysis then future work may look into using computer vision techniques to translate paper models into computer models for analysis. To study HC-1 and HC-2, we define three treatments: (TC-Paper) subjects construct and review goal models on paper; (TC-Tool) subjects construct and review goal models in a tool; and (TC-Auto) subjects specify sufficient information to enable for the construction of a goal model and stakeholders review the constructed goal model. These three treatments allow us to explicitly learn what utility there is in each process but by comparing TC-Paper and TC-Tool with TC-Auto, we can understand if simplification or automation can provide similar benefits. While other hypotheses and treatments may exist, we use these in our study discussed in Section 8.4.2. Analysis. Measuring the utility of GORE analysis is entirely dependent on the type of analysis chosen for the measurement. In this subsection, we focus on some assumptions that are general and exist across approaches. These assumptions may apply to both the initial analysis in evolutionary approaches as well as the specialized analysis in domain specific extensions (e.g., security, law). We propose the following research questions to examine analysis assumptions and motivate our work:

• What insights do participants gain by assigning initial evaluation labels (or labels for analysis methods)? • How sufficient is a textual representation for capturing initial evaluation labels (or labels for analysis methods)? • What utility is there in using analysis for answering questions, debugging, and model comprehension? • What utility does each technique provide (e.g., forward analysis, backward analysis, metrics over models [44][103])? • For each technique, to what extent are there benefits in performing analysis manually vs. automatically? • To what extent do stakeholders have to understand the underlying analysis to benefit from it? • How likely is it that stakeholders use the analysis to justify their preferred answer rather than the best result? • To what degree is there utility in completing analysis manually on paper?

In the motivating example, we can ask: To what extent was asking “what if” questions useful in discovering goals and relationships that were missing from the model? To understand analysis deeper, we propose three hypotheses: (HA-1) There is no utility in using analysis techniques for debugging and model comprehension activities. (HA-2) There is no utility in using analysis techniques for answering questions. (HA-3) There is no additional utility in using interactive analysis techniques over automated techniques. HA-3 specifically focuses on understanding automation opportunities in analysis. When examining these hypotheses, evidence and conclusions gained from studying one technique may not be valid with other techniques. Since there is no unified GORE process and individuals are left to interpret how to proceed in analysis, it is unclear whether stakeholders generate their questions as they are reviewing the model or know them a priori. In the motivating example, did the stakeholders intend to explore the impacts of Permanent Construction, or was it only a result of making connections from the Chapter 8. Conclusions & Future Work 137 model? The assumption is made that stakeholders can translate their problem into a form appropriate for goal model analysis techniques. How does the City in the Bike Lanes example go from being curious about Permanent Construction to formulating a forward analysis appropriate input? With this additional assumption, we consider two treatments for evaluating these hypotheses: (TA-Oz) subjects ask questions in natural language and are returned results; and (TA-Self) subjects are trained on the analysis method and must ask questions and perform analysis independently. The treatments and hypotheses are further described in Section 8.4.2. Evolution. Evolution activities assume that not all changes are known in the early stages of a project; thus, stakeholders need to be able to update models and deal with unanticipated changes. We define an evolution activity to be any activity that involves reviewing goal models or analysis results for the purpose of updating/reviewing past decisions or monitoring current conditions. With this definition, we consider the following research questions as motivation:

• To what degree is reviewing past analysis results helpful in understanding current issues? • To what degree is reviewing models helpful in making decisions in the present? • To what degree is making projections about the future meaningful for project outcomes?

In the motivating example, we can ask: To what extent is it helpful to make predictions about how variations in Bike Lane Usage will aﬀect other goals? We propose two initial hypotheses to explore evolution activities: (HE-1) There is no utility in monitoring goal model elements later in the project lifecycle. (HE-2) There is no utility in reviewing past analysis results in understanding current decisions. Addressing HE-1 is outside the scope of our study and should be considered through the lens of action research. If we reject HE-2, then there are automation opportunities for storing and retrieving past models, analysis results, and conclusions. We propose two treatments to evaluate HE-2: (TE-Model) subjects are given past goal models and analysis; and (TE-Info) subjects are given past questions and decisions. We discuss how these treatments are used in our study design in Section 8.4.2.

8.4.2 Study Design

In this section, we propose a series of empirical studies aimed to study the validity of assumptions made by the GORE community with respect to the utility of the approach. Overarching Tenets. Prior to introducing our study protocol, we discuss overarching tenets that guide our design. Stakeholders’ Perceptions: Utility, as a construct, is a perceptual judgement of each stakeholder. In order to judge the true utility of something, it must not be evaluated in a hypothetical context. It is not necessarily the case that all studies must be completed In vivo. We argue that In vitro studies in a laboratory can produce valuable results if stakeholders bring their own problem or project into the lab. Stakeholders must be domain experts in the problem they model and provide their own analysis questions. Comparing Perceptions: It is difficult to compare individual perceptions of utility. In between-subject experiments, we can establish baselines of expertise and experience for comparing subjects and have them perform identical tasks; however, we cannot make claims that subjects would have the same perception of alternative treatments. One mitigating suggestion is to have teams of subjects split into treatment groups and then compare activities of the group as a whole. In within-subject experiments, we can give Chapter 8. Conclusions & Future Work 138 both treatments to the same subject; however, we cannot make guarantees that the subject’s perception will not be confounded by the differences between tasks. One mitigating suggestion is to have subjects with large projects split the project into equal sub-projects for each treatment, but this doesn’t mitigate the presence of a learning effect. The silver lining in our study is that we are not yet trying to prove that a specific utility exists. We are focusing on discovering potential benefits, so false positives, while undesirable, may be tolerated, and false negatives can best be mitigated through replications. Remove Modelers: In order for goal modeling to reach wide spread adoption, we need to remove expert modelers from the process. Stakeholders should be able to gain value from the activities without the presence of expert modelers. Studies may need to have a training portion where subjects are given background to engage in goal modeling tasks but should not have expert modelers completing tasks. The exception to this tenet is if expert modelers are used in a ‘Wizard of Oz’ experiment [102] to mimic some automated process. If utility is found without the immediate presence of expert modelers, then there may be value in automating modeling and analysis processes, as well as investigating how to use the modeling as a service paradigm. We would also need to investigate the degree to which stakeholders trust any automation. Scale Matters: Goal model studies use small models. The utility of the GORE approach could be dependent on model size. Perhaps construction of large models is hard, reducing the utility, but automated analysis of large models would be useful if tooling was sufficiently scaleable. We need to investigate how the value of automated processes depends on model size, as well as the interaction between model size and the cognitive load required to understand models as a whole and the results of analysis procedures. Results from all of these studies must be considered in the context of model size. Common Design Components. Our protocol consists of ordered study components for each activity: construction, analysis, and evolution. The study for each activity, taken together, will give us a full picture of the utility of GORE. For consistency between study replications and to enable longitudinal studies, we plan to collect common data. At the beginning of each study, we must document, in the subjects’ own words, a description of the problem and domain which will be explored with the GORE approach. By documenting this up front, it can be used in later phases of the study as well as in meta analysis. We recommend using a common template to structure the subjects’ descriptions. Prior to analysis observations, subjects must be asked whether they have sufficiently modeled their problem/domain? If not, what is missing and why? When subjects are evaluated across multiple activities, subjects would be asked which activity they found as having the most value in understanding their problem/domain. In order to evaluate evolutionary questions, all models and results must be collected and stored. The stakeholders’ reasoning and final decision would also be documented. Construction. For studying construction (i.e., HC-1 and HC-2), we propose two complementary approaches. The first study is to evaluate the construction of models within subjects (Const-Study-Within). This is most appropriate when a single stakeholder with a large project to model is found. The project is then divided up into three parts, and each part is applied to a treatment (i.e., TC-Paper, TC-Tool, and TC-Auto) in random order. This can be done as a single case study or as a quasi-experiment with a set of stakeholders with independent projects. It is important that the modeler reflects on each treatment separately prior to comparing treatments. The second study is to evaluate the construction of models between subjects (Const-Study-Between). This can be completed as a controlled experiment where a baseline can be achieved about the subjects’ domain expertise and modeling abilities (for comparing TC-Paper and TC-Tool). The second study is recommended when groups of stakeholders are working Chapter 8. Conclusions & Future Work 139 on the same project or problem. The group can be randomly divided into one of the three treatment groups to draw separate models. Each subject can reflect on what insights and value the treatment provided and then as a team they can compare and merge their models. While an investigation of the model merge process would be beneficial, it is currently outside the scope of this work. We chose these designs over focusing on a single treatment and asking subjects what they found useful, as is done in many case studies. This design mitigates hypothesis guessing and reduces subjects’ desire to please the researcher. This study protocol is process agnostic. We are not advocating the use of any method for automatically constructing goal models, but rather explore the potential benefits. We envision TC-Auto being implemented either by having GORE experts construct the models (i.e., Wizard of Oz), or using natural-language processing or other AI techniques to construct them.

Analysis. For initial studies of the utility of analysis, we propose a single within subjects (Analysis- Study-Initial) study, where each subject completes both treatments (i.e., TA-Oz and TA-Self). This is not an experiment, but will act as an exploration to the utility of analysis. Each question the subject wants to ask using the model and analysis technique would be documented. Then, depending on the question, an appropriate analysis technique is chosen and the analysis result is produced. The subject is then asked a series of questions to assess the assumptions of analysis: Did the analysis result in changes or improvements to the model? Did the subject successfully answer the question they originally posed, if so, how many analysis iterations did it require? This study will not give conclusive evidence for HA-1, HA-2, or HA-3, but it will give insights into how to directly test these hypotheses. When subjects complete both treatments (i.e., TA-Oz and TA- Self), deception can be used to make the subjects believe that the goal of the study is to compare techniques, mitigating any performance bias, since the study collects data for both. By asking the subjects to compare techniques, we can investigate what barriers exist but also how deeply the subjects engaged with the analysis and whether they ﬁnd it believable. We also plan to compare our observations of the results with the subjects’ analysis of them. This study can be replicated with individual analysis techniques.

Evolution. In an exploratory evaluation of HE-2, subjects may be recalled after several weeks or even months have passed. Subjects are asked to review the question and domain they originally considered and to recall what conclusions they reached. We then reveal the subjects’ original conclusions. Subjects are then placed into one of the two treatments groups (i.e., TE-Model or TE-Info) and asked to repeat a modified treatment from either a construction or analysis activity. Where two time-delayed interventions are possible, subjects can be evaluated with both treatments (i.e., TE-Model and TE-Info). Note that we give only a brief overview of the HE-2 study here because it can only be addressed in the contexts of a longitudinal study, where participants revisit the exploration they previously completed. This area is the most uncertain as to the benefits, but also the validity of such a study is most at risk to confounding variables from other research questions. If we were to study this process by itself, we suspect the results would be influenced by the study design setup, and hence believe it is imperative to first understand the effect of earlier modeling and analysis phases.

Potential Threats to Validity. While not all threats to validity for this research can be known at this time, we explore some of them [144]. Conclusion Validity: These studies are at risk of the reliability of measurement threat, and we hope to form a community of reviewers for improved measurement of utility in this protocol. When Chapter 8. Conclusions & Future Work 140 studies are completed between-subjects, an argument must be made, by the individual study authors, to mitigate the random heterogeneity of subjects threat. Low sample size may threaten the conclusions of the experiments in Section 8.4.2. Construct Validity: Whenever combinations of construction, analysis, and evolution are evaluated with the same subjects, there is an interaction of different treatments effect. This threat can be mitigated through replications with different combinations of treatment protocols. The experimenter expectancies threat should be mitigated by using explicit question wording in protocols whenever possible and documenting off-script questions that may bias subjects. Internal Validity: We must take care to mitigate the selection, compensatory rivalry and resentful demoralization threats by adequately analyzing subjects’ prior knowledge and views of GORE approaches prior to the study as well as evaluating their motivations for participating in the study. Proponents and opponents of goal modeling should be avoided in such studies. External Validity: This line of research is motivated by external validity issues in past GORE studies. Each individual study will have the interaction of selection and treatment and interaction of setting and treatment threats, but it is our goal that all studies taken together can mitigate these threats by using different populations and settings, forming a body of knowledge about the utility of GORE approaches. To mitigate these threats, each study should document their context of observation (i.e., population demographics, model domain and size, as well as the specific GORE approach/languages used). These studies were crafted from the lens of the iStar family of GORE approaches. We believe that at least the modeling portion of this methodology directly applies to other GORE approaches, such as KAOS, and that the study as a whole can be easily adapted for other approaches.

8.4.3 Summary

We reflect on some of the underlying assumptions made in the GORE literature and how these assumptions may have contributed to the lack of adoption of GORE approaches in industry. We divided GORE activities into the categories of construction, analysis, and evolution to generate assumptions that might interact with understanding of the utility of GORE approaches, and connected these with testable hypotheses and treatments. We propose a study protocol consisting of a series of studies to evaluate where modelers find benefits in GORE approaches and explore opportunities for process automation. Bibliography

[1] i* Wiki: i* Tools. http://istar.rwth-aachen.de/tiki-index.php?page=i%2A+Tools, 2015. Accessed: 2015-02-20.

[2] i* Wiki: i*-related PhD Dissertations. http://istar.rwth-aachen.de/tiki-index.php?page= i%2A-related+Phd+Dissertations, 2018. Accessed: 2018-03-14.

[3] Zahra Shakeri Hossein Abad, Mohammad Noaeen, and Guenther Ruhe. Requirements Engineering Visualization: A Systematic Literature Review. In 2016 IEEE 24th International Requirements Engineering Conference (RE), pages 6–15, September 2016.

[4] Catarina Almeida, Miguel Goulão,and JoãoAraújo. A Systematic Comparison of i* Modelling Tools Based on Syntactic and Well-formedness Rules. In Proceedings of 6th International I* Workshop (iStar’13), pages 43 – 48, 2013.

[5] Daniel Amyot. Introduction to the User Requirements Notation: Learning by Example. Computer Networks, 42(3):285–301, June 2003.

[6] Daniel Amyot, Sepideh Ghanavati, Jennifer Horkoﬀ, Gunter Mussbacher, Liam Peyton, and Eric Yu. Evaluating Goal Models Within the Goal-Oriented Requirement Language. International Journal of Intelligent Systems, 25(8):841–877, 2010.

[7] Daniel Amyot, Azalia Shamsaei, Jason Kealey, Etienne Tremblay, Andrew Miga, Gunter Muss- bacher, Mohammad Alhaj, Rasha Tawhid, Edna Braun, and Nick Cartwright. Towards Advanced Goal Model Analysis with jUCMNav. In Proceedings of the International Conference on Conceptual Modeling (ER’12), pages 201–210, 2012.

[8] James A. Anderson. Automata Theory with Modern Applications. Cambridge University Press, 2006.

[9] Annie Anton and Colin Potts. The Use of Goals to Surface Requirements for Evolving Systems. In Proceedings of the 20th International Conference on Software Engineering (ICSE’98), pages 157–166, 1998.

[10] Aprajita. TimedGRL: Specifying Goal Models Over Time. Master’s thesis, McGill University, 2017.

[11] Aprajita, Sahil Luthra, and Gunter Mussbacher. Specifying Evolving Requirements Models with TimedURN. In Proceedings of the Ninth International Workshop on Modeling in Software Engi- neering, pages 26–32, May 2017.

141 Bibliography 142

[12] Aprajita and Gunter Mussbacher. TimedGRL: Specifying Goal Models Over Time. In Proceedings of the the Sixth International Workshop on Model-Driven Requirements Engineering (MoDRE’16), 2016.

[13] Christel Baier and Joost-Pieter Katoen. Principles of Model Checking. MIT Press, 2008.

[14] Luciano Baresi, Liliana Pasquale, and Paola Spoletini. Fuzzy Goals for Requirements-Driven Adaptation. In Proceedings of the 18th IEEE International Requirements Engineering Conference (RE’10), pages 125–134, 2010.

[15] Nelly Bencomo, Jon Whittle, Pete Sawyer, Anthony Finkelstein, and Emmanuel Letier. Require- ments Reﬂection: Requirements As Runtime Entities. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering (ICSE’10), pages 199–202, 2010.

[16] Edward H. Bersoﬀ. Elements of Software Conﬁguration Management. IEEE Transactions on Software Engineering, 10(1):79–87, January 1984.

[17] Benjamin Blazy, Ashlee DeLine, Benjamin Frey, and Maxwell Miller. Software Requirements Speciﬁcation (SRS) Lane Management System. CSE 435 Project, Michigan State University, 2014.

[18] Barry W. Boehm. Software Risk Management: Principles and Practices. IEEE Software, 8(1):32– 41, January 1991.

[19] Paolo Bresciani, Anna Perini, Paolo Giorgini, Fausto Giunchiglia, and John Mylopoulos. Tropos: An Agent-Oriented Software Development Methodology. Autonomous Agents and Multi-Agent Systems, 8(3):203–236, May 2004.

[20] Frank R. Burton. Trade-oﬀs in system of systems acquisition. PhD thesis, University of York, 2014.

[21] Carlos Cares, Xavier Franch, Anna Perini, and Angelo Susi. Towards Interoperability of I* Models Using iStarML. Computer Standards & Interfaces, 33(1):69–79, January 2011.

[22] Juan Pablo Carvallo and Xavier Franch. Building Strategic Enterprise Context Models with i*: A Pattern-Based Approach. In Stephan Aier, Mathias Ekstedt, Florian Matthes, Erik Proper, and Jorge L. Sanz, editors, Trends in Enterprise Architecture Research and Practice-Driven Research on Enterprise Transformation, pages 40–59, Berlin, Heidelberg, 2012. Springer Berlin Heidelberg.

[23] Betty H. C. Cheng, Rog´eriode Lemos, Holger Giese, Paola Inverardi, and Jeﬀ Magee. Software Engineering for Self-Adaptive Systems. Lecture Notes in Computer Science, 5525, 2009.

[24] Christopher Cheong and Michael Winikoﬀ. Hermes: A Methodology for Goal Oriented Agent Interactions. In Proceedings of the 4th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS’05), pages 1121–1122, 2005.

[25] Christopher Cheong and Michael Winikoﬀ. Hermes: Designing Goal-Oriented Agent Interactions. In Proceedings of the International Workshop on Agent-Oriented Software Engineering (AOSE’05), LNCS 3950, pages 16–27, 2006. Bibliography 143

[26] Lawrence Chung, Brian A. Nixon, Eric Yu, and John Mylopoulos. Non-Functional Requirements in Software Engineering. Kluwer Academic Publishers, Norwell, MA, 2000.

[27] Edmund M. Clarke, E. Allen Emerson, and Joseph Sifakis. Model Checking: Algorithmic Veriﬁ- cation and Debugging. Commun. ACM, 52(11):74–84, November 2009.

[28] Edmund M. Clarke, E. Allen Emerson, and A. Prasad Sistla. Automatic verification of finite- state concurrent systems using temporal logic specifications. ACM Transactions on Programming Languages and Systems (TOPLAS), 8(2):244–263, April 1986.

[29] Andreas Classen, Patrick Heymans, Pierre-Yves Schobbens, Axel Legay, and Jean-Fran¸coisRaskin. Model Checking Lots of Systems: Eﬃcient Veriﬁcation of Temporal Properties in Software Product Lines. In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering (ICSE’10), pages 335–344, 2010.

[30] Paul C. Clements and Linda Northrop. Software Product Lines: Practices and Patterns. Addison- Wesley, 2001.

[31] Client IO. JointJS/Rappid - HTML 5 Diagramming Toolkit. http://jointjs.com/rappid/tour, 2016. Accessed: 2016-02-21.

[32] Timothy J. Colton. Big Daddy: Frederick G. Gardiner and the building of Metropolitan Toronto. University of Toronto Press, 1980.

[33] Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Cliﬀord Stein. Introduction To Algorithms. MIT Press, 2001.

[34] Fabiano Dalpiaz, Alexander Borgida, Jennifer Horkoﬀ, and John Mylopoulos. Runtime Goal Models: Keynote. In Proceedings of the 7th International Conference on Research Challenges in Information Science (RCIS’13), pages 1–11, 2013.

[35] Fabiano Dalpiaz, Xavier Franch, and Jennifer Horkoﬀ. iStar 2.0 Language Guide. arXiv:1605.07767, 2016.

[36] Adnan Darwiche and Judea Pearl. On the Logic of Iterated Belief Revision. In Proceedings of the 5th Conference on Theoretical Aspects of Reasoning About Knowledge (TARK’94), pages 5–23, 1994.

[37] Gregorio Diaz, Elena Navarro, Maria-Emilia Cambronero, Valentin Valero, and Fernando Cuartero. Testing Time Goal-Driven Requirements with Model Checking Techniques. In Proceedings of the 14th IEEE International Conference and Workshops on the Engineering of Computer-Based Systems (ECBS’07), pages 503–514, March 2007.

[38] Olive J. Dunn. Multiple Comparisons Using Rank Sums. Technometrics, 6(3):241–252, 1964.

[39] Allen H. Dutoit and Barbara Paech. Rationale Management in Software Engineering. In S.K. Chang, editor, Handbook of Software Engineering and Knowledge Engineering, volume 1. World Scientiﬁc, 2001.

[40] Amy Edmondson. Psychological Safety and Learning Behavior in Work Teams. Administrative Science Quarterly, 44(2):350–383, 1999. Bibliography 144

[41] Neil A. Ernst, Alexander Borgida, and Ivan Jureta. Finding Incremental Solutions for Evolving Requirements. In Proceedings of the 19th IEEE International Requirements Engineering Conference (RE’11), pages 15–24, 2011.

[42] Michail Famelis. Managing Design-Time Uncertainty in Software Models. PhD thesis, University of Toronto, June 2016.

[43] Jay W. Forrester. Industrial Dynamics. Pegasus Communications, 1961.

[44] Xavier Franch. A Method for the Deﬁnition of Metrics over i* Models. In Proceedings of the International Conference on Advanced Information Systems Engineering (CAiSE’09), pages 201– 215, 2009.

[45] Xavier Franch, Daniel MéndezFernández,Marc Oriol, Andreas Vogelsang, Rogardt Heldal, Eric Knauss, Guilherme Horta Travassos, Jeffrey C. Carver, Oscar Dieste, and Thomas Zimmermann. How do Practitioners Perceive the Relevance of Requirements Engineering Research? An Ongoing Study. In In Proceedings of the 25th IEEE International Requirements Engineering Conference (RE’17), pages 382–387, 2017.

[46] Xavier Franch, Ron S. Kenett, Fabio Mancinelli, Angelo Susi, David Ameller, Maria Carmela Annosi, Ron Ben-Jacob, Yehuda Blumenfeld, O. H. Franco, Daniel Gross, Lidia L´opez, Mirko Morandini, Marc Oriol, and Alberto Siena. The RISCOSS Platform for Risk Management in Open Source Software Adoption. In Proceedings of the IFIP International Conference on Open Source Systems (OSS’15), pages 124–133, 2015.

[47] G¨unter Gans, Matthias Jarke, Gerhard Lakemeyer, and Dominik Schmitz. Deliberation in a Modeling and Simulation Environment for Inter-Organizational Networks. In Proceedings of the International Conference on Advanced Information Systems Engineering (CAiSE’03), pages 242– 257, 2003.

[48] G¨unter Gans, Matthias Jarke, Gerhard Lakemeyer, and Dominik Schmitz. Deliberation in a Metadata-Based Modeling and Simulation Environment for Inter-Organizational Networks. Infor- mation Systems, 30(7):587–607, 2005.

[49] Malik Ghallab, Dana Nau, and Paolo Traverso. Automated Planning: Theory and Practice, chapter Constraint Satisfaction Techniques. The Morgan Kaufmann Series in Artiﬁcial Intelligence. Elsevier Science, 2004.

[50] Tom Gilb. Principles of Software Engineering Management. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1988.

[51] Paolo Giorgini, John Mylopoulos, Eleonora Nicchiarelli, and Roberto Sebastiani. Reasoning with Goal Models. In Proceedings of the International Conference on Conceptual Modeling (ER’02), pages 167–181, 2002.

[52] Paolo Giorgini, John Mylopoulos, Eleonora Nicchiarelli, and Roberto Sebastiani. Formal Reasoning Techniques for Goal Models. Journal on Data Semantics, 1:1–20, 2003. Bibliography 145

[53] Paolo Giorgini, John Mylopoulos, and Roberto Sebastiani. Goal-oriented Requirements Analysis and Reasoning in the Tropos Methodology. Engineering Applications of Artiﬁcial Intelligence, 18(2):159–171, 2005.

[54] Michael W. Godfrey and Daniel M. German. The Past, Present, and Future of Software Evolution. In Proceedings of the Frontiers of Software Maintenance (FoSM’08), pages 129–138, September 2008.

[55] Hassan Gomaa. Designing Software Product Lines with UML: From Use Cases to Pattern-Based Software Architectures. Addison-Wesley, 2004.

[56] Ricardo Gon¸calves, Matthias Knorr, Jo˜aoLeite, and Martin Slota. Non-monotonic Temporal Goals. In Proceedings of the International Conference on Logic Programming and Nonmonotonic Reasoning (LPNMR’13), pages 374–386, 2013.

[57] Susanne Graf and Andreas Prinz. Time in State Machines. Fundamenta Informaticae, 77(1-2):143– 174, 2007.

[58] Alicia M. Grubb. Adding Temporal Intention Dynamics to Goal Modeling: A Position Paper. In Proceedings of the 7th International Workshop on Modeling in Software Engineering (MiSE’15), 2015.

[59] Alicia M. Grubb. Reﬂection on Evolutionary Decision Making with Goal Modeling via Empirical Studies. In Proceedings of the 26th IEEE International Requirements Engineering Conference (RE’18), pages 376–381, 2018.

[60] Alicia M. Grubb and Marsha Chechik. Looking into the Crystal Ball: Requirements Evolution over Time. In Proceedings of the 24th IEEE International Requirements Engineering Conference (RE’16), 2016.

[61] Alicia M. Grubb and Marsha Chechik. Modeling and Reasoning with Changing Intentions: An Experiment. In Proceedings of the 25th IEEE International Requirements Engineering Conference (RE’17), 2017.

[62] Alicia M. Grubb and Marsha Chechik. BloomingLeaf: A Formal Tool for Requirements Evolution over Time. In Proceedings of the 26th IEEE International Requirements Engineering Conference (RE’18): Posters & Tool Demos, pages 490–491, 2018.

[63] Alicia M. Grubb and Marsha Chechik. Reconstructing the Past: The Case of the Spadina Ex- pressway. Submitted, 2018.

[64] Alicia M. Grubb, Gary Song, and Marsha Chechik. GrowingLeaf: Supporting Requirements Evo- lution over Time. In Proceedings of 9th International I* Workshop (iStar’16), pages 31–36, 2016.

[65] James D. Hart. Discovering System Dynamics in Software Engineering: Foundations for the Software Learning Organization. Software Process Dynamics LLC, USA, 2008.

[66] Thomas Hartmann, Francois Fouquet, Gregory Nain, Brice Morin, Jacques Klein, Olivier Barais, and Yves Le Traon. A Native Versioning Concept to Support Historized Models at Runtime. In Proceedings of the International Conference on Model Driven Engineering Languages and Systems (MODELS’14), pages 252–268, 2014. Bibliography 146

[67] John E. Hopcroft and Jeﬀrey D. Ullman. Introduction to Automata Theory, Languages, and Computation. Addison-Wesley Series in Computer Science and Information Processing. Addison- Wesley, 1979.

[68] Jennifer Horkoﬀ. Iterative, Interactive Analysis of Agent-Goal Models for Early Requirements Engineering. PhD thesis, University of Toronto, 2012.

[69] Jennifer Horkoﬀ, Fatma Basak Aydemir, Evellin Cardoso, Tong Li, Alejandro Mat´e,Elda Paja, Mattia Salnitri, John Mylopoulos, and Paolo Giorgini. Goal-Oriented Requirements Engineer- ing: A Systematic Literature Map. In Proceedings of the 24th IEEE International Requirements Engineering Conference (RE’16), pages 106–115, 2016.

[70] Jennifer Horkoﬀ, James Lockerbie, Xavier Franch, Eric Yu, and John Mylopoulos. Report on the First International i* Teaching Workshop (iStarT). ACM SIGSOFT Software Engineering Notes, 40(6):24–27, 2015.

[71] Jennifer Horkoﬀ and Neil A. M. Maiden. Creative Leaf: A Creative iStar Modeling Tool. In Proceedings of 9th International I* Workshop (iStar’16), pages 25–30, 2016.

[72] Jennifer Horkoﬀ, Neil A. M. Maiden, and David Asboth. Creative Goal Modeling for Innovative Requirements, 2018. Submitted.

[73] Jennifer Horkoﬀ and Eric Yu. A Qualitative, Interactive Evaluation Procedure for Goal-and Agent- oriented Models. In Proceedings of the CAiSE’09 Forum at the 21th International Conference on Advanced Information Systems Engineering (CAiSE’09), pages 19–24, 2009.

[74] Jennifer Horkoﬀ and Eric Yu. Finding Solutions in Goal Models: An Interactive Backward Reason- ing Approach. In Proceedings of the 18th IEEE International Requirements Engineering Conference (RE’10), pages 59–75, 2010.

[75] Jennifer Horkoﬀ and Eric Yu. Analyzing Goal Models: Diﬀerent Approaches and How to Choose Among Them. In Proceedings of the 2011 ACM Symposium on Applied Computing (SAC’11), pages 675–682, 2011.

[76] Jennifer Horkoﬀ and Eric Yu. Comparison and Evaluation of Goal-Oriented Satisfaction Analysis Techniques. Requirements Engineering, 18(3):199–222, 2013.

[77] Jennifer Horkoﬀ and Eric Yu. Interactive Goal Model Analysis For Early Requirements Engineer- ing. Requirements Engineering, 21(1):29–61, 2016.

[78] Jennifer Horkoﬀ, Yijun Yu, and Eric Yu. OpenOME: An Open-source Goal and Agent-Oriented Model Drawing and Analysis Tool. In Proceedings of 5th International I* Workshop (iStar’11), pages 154–156, 2011.

[79] Shareeful Islam and Siv Hilde Houmb. Integrating Risk Management Activities into Require- ments Engineering. In Proceedings of the 4th International Conference on Research Challenges in Information Science (RCIS’10), pages 299–310, 2010.

[80] Michael Jackson. The World and the Machine. In Proceedings of the 17th International Conference on Software Engineering (ICSE’95), pages 283–292, 1995. Bibliography 147

[81] Paul Johannesson. The Role of Business Models in Enterprise Modelling. In John Krogstie, Andreas Lothe Opdahl, and Sjaak Brinkkemper, editors, Conceptual Modelling in Information Systems Engineering, pages 123–140. Springer Berlin Heidelberg, 2007.

[82] T. Capers Jones. Estimating Software Costs. McGraw-Hill, Inc., Hightstown, NJ, USA, 1998.

[83] Oliver Karras, Stephan Kiesling, and Kurt Schneider. Supporting Requirements Elicitation by Tool-Supported Video Analysis. In Proceedings of the 24th IEEE International Requirements Engineering Conference (RE’16), pages 146–155, September 2016.

[84] Robert M. Keller. Formal Veriﬁcation of Parallel Programs. Communication of the ACM, 19(7):371–384, July 1976.

[85] JörgKienzle, Nicolas Guelfi, and Sadaf Mustafiz. Crisis Management Systems: A Case Study for Aspect-Oriented Modeling. In Transactions on Aspect-Oriented Software Development VII, pages 1–22. Springer, 2010.

[86] David R. Krathwohl. A Revision of Bloom’s Taxonomy: An Overview. Journal of Theory Into Practice, 41(4):212–218, Autumn 2002.

[87] Charles W. Krueger. Leveraging Integrated Model-Driven Development and Software Product Line Development Technologies. In Proceedings of the ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages and Applications (OOPSLA’07), Companion Volume, pages 836–837, 2007.

[88] Krzysztof Kuchcinski and Radoslaw Szymanek. JaCoP - Java Constraint Programming solver. http://jacop.osolpro.com, 2016. Accessed: 2016-02-21.

[89] Timothy C. Lethbridge, Susan Elliott Sim, and Janice Singer. Studying Software Engineers: Data Collection Techniques for Software Field Studies. Empirical Software Engineering, 10(3):311–341, July 2005.

[90] Emmanuel Letier. Reasoning about Agents in Goal-Oriented Requirements Engineering. PhD thesis, Universite Catholique de Louvain, Belgium, 2001.

[91] Emmanuel Letier, Jeﬀ Kramer, Jeﬀ Magee, and Sebastian Uchitel. Fluent Temporal Logic for Discrete-time Event-based Models. ACM SIGSOFT Software Engineering Notes, 30(5):70–79, 2005.

[92] Emmanuel Letier and Axel van Lamsweerde. Deriving Operational Software Speciﬁcations from System Goals. In Proceedings of the 10th ACM SIGSOFT Symposium on Foundations of Software Engineering (FSE’02), pages 119–128, 2002.

[93] Tong Li, Jennifer Horkoﬀ, and John Mylopoulos. A Prototype Tool for Modeling and Analyzing Security Requirements from A Holistic Viewpoint. In Proceedings of the International Conference on Advanced Information Systems Engineering (CAiSE’14): Forum/Doctoral Consortium, pages 185–192, 2014.

[94] Sotirios Liaskos, Alexei Lapouchnian, Yijun Yu, Eric Yu, and John Mylopoulos. On Goal-based Variability Acquisition and Analysis. In 14th IEEE International Requirements Engineering Con- ference (RE’06), pages 79–88, September 2006. Bibliography 148

[95] Bennet P Lientz, E. Burton Swanson, and Gail E Tompkins. Characteristics of Application Soft- ware Maintenance. Communications of the ACM, 21(6):466–471, June 1978.

[96] David Lo, Nachiappan Nagappan, and Thomas Zimmermann. How Practitioners Perceive the Rel- evance of Software Engineering Research. In Proceedings of the 10th Joint Meeting on Foundations of Software Engineering (FSE’15), pages 415–425, 2015.

[97] Lidia L´opez and Xavier Franch. Applying Business Strategy Models in Organizations. In Proceed- ings of the 7th International I* Workshop (iStar’14), 2014.

[98] Hao Luo and Daniel Amyot. Towards a Declarative, Constraint-Oriented Semantics with a Generic Evaluation Algorithm for GRL. Proceedings of 5th International I* Workshop (iStar’11), 2011.

[99] Sahil Luthra, Aprajita, and Gunter Mussbacher. Visualizing Evolving Requirements Models with TimedURN. In Proceedings of the 10th International Workshop on Modelling in Software Engi- neering (MiSE’18), 2018.

[100] Raymond J. Madachy. Software Process Dynamics. Wiley, 2007.

[101] M. Morris Mano. Digital Design. Prentice Hall, 2002.

[102] Bella Martin and Bruce Hanington. Universal Methods of Design. Rockport Publishers, 2012.

[103] George Mathew, Tim Menzies, Neil Ernst, and John Klein. Shorter Reasoning About Larger Requirements Models. In Proceedings of the 25th IEEE International Requirements Engineering Conference (RE’17), 2017.

[104] Alistair Mavin, Philip Wilkinson, Sabine Teuﬂ, Henning Femmer, Jonas Eckhardt, and Jakob Mund. Does Goal-Oriented Requirements Engineering Achieve Its Goal? In In Proceedings of the 25th IEEE International Requirements Engineering Conference (RE’17), pages 174–183, Septem- ber 2017.

[105] Dennis L. Meadows, William W. Behrens, Donella H. Meadows, Roger F. Naill, Jørgen Randers, and Erich K. O. Zahn. Dynamics of Growth in a Finite World. Wright-Allen Press, 1974.

[106] Tom Mens. A State-of-the-Art Survey on Software Merging. IEEE Transactions on Software Engineering, 28(5):449–462, May 2002.

[107] Tom Mens and Serge Demeyer. Software Evolution (1st Edition). Springer Publishing Company, Incorporated, 2008.

[108] Robin Milner. A Calculus of Communicating Systems. LNCS, 92, 1980.

[109] Felix Naumann, Johann-Christoph Freytag, and Ulf Leser. Completeness of Information Sources. In Proceedings of the Workshop on Data Quality in Cooperative Information Systems (DQCIS’03), 2003.

[110] Chi Mai Nguyen, Roberto Sebastiani, Paolo Giorgini, and John Mylopoulos. Multi-objective Reasoning with Constrained Goal Models. Requirements Engineering, December 2016. Bibliography 149

[111] Chi Mai Nguyen, Roberto Sebastiani, Paolo Giorgini, and John Mylopoulos. Modeling and Rea- soning on Requirements Evolution with Constrained Goal Models. In Proceedings of the 15th International Conference Software Engineering and Formal Methods (SEFM’17), pages 70–86, 2017.

[112] OED Online. “utility, n.”. Oxford University Press, 2018. www.oed.com/view/Entry/220771. Accessed: 2018-03-13.

[113] Peyman Oreizy, Nenad Medvidovic, and Richard N. Taylor. Architecture-based Runtime Software Evolution. In Proceedings of the 20th International Conference on Software Engineering (ICSE’98), pages 177–186, 1998.

[114] Mark Osbaldeston. Unbuilt Toronto: A History of the City That Might Have Been. The City That Might Have Been. Dundurn, 2008.

[115] Elda Paja, Alejandro Mat´e,Carson Woo, and John Mylopoulos. Can Goal Reasoning Techniques Be Used for Strategic Decision-Making? In Proceedings of the 35th International Conference on Conceptual Modeling (ER’16), pages 530–543, 2016.

[116] I. Porres Paltor and Johan Lilius. Digital Sound Recorder: A Case Study on Designing Embedded Systems Using the UML Notation. Turku Centre for Computer Science, 1999.

[117] Jo˜aoPimentel and Jaelson Castro. piStar Tool – A Pluggable Online Tool for Goal Modeling. In Proceedings of the 26th IEEE International Requirements Engineering Conference (RE’18): Posters & Tool Demos, 2018.

[118] Klaus Pohl, Guenter Boeckle, and Frank van der Linden. Software Product Line Engineering : Foundations, Principles, and Techniques. Springer, 2005.

[119] Gil Regev and Alain Wegmann. Where Do Goals Come From: The Underlying Principles of Goal- Oriented Requirements Engineering. In Proceedings of the 13th IEEE International Requirements Engineering Conference (RE’05), pages 353–362, 2005.

[120] George P. Richardson. Problems with Causal-loop Diagrams. System Dynamics Review, 2(2):158– 170, 1986.

[121] William N. Robinson. A Requirements Monitoring Framework for Enterprise Systems. Require- ments Engineering, 11(1):17–41, December 2005.

[122] Colette Rolland. Goal Oriented Requirements Engineering. In Marite Kirikova, Janis Grund- spenkis, Wita Wojtkowski, W. Gregory Wojtkowski, Stanislaw Wrycza, and JoˇzeZupanˇciˇc,editors, Information Systems Development: Advances in Methodologies, Components, and Management, pages 35–51. Springer US, 2002.

[123] Colette Rolland, Carine Souveyet, and Camille Ben Achour. Guiding Goal Modeling Using Sce- narios. IEEE Transactions on Software Engineering, 24(12):1055–1071, 1998.

[124] Albert Rose. Governing Metropolitan Toronto: A Social and Political Analysis, 1953-1971. Insti- tute of Governmental Studies, 1972. Bibliography 150

[125] James Rumbaugh, Ivar Jacobson, and Grady Booch. The Uniﬁed Modeling Language Reference Manual (2nd Edition). Pearson Higher Education, 2004.

[126] Per Runeson, Martin H¨ost,Austen Rainer, and Bj¨ornRegnell. Case Study Research in Software Engineering - Guidelines and Examples. Wiley, 2012.

[127] Richard P. Runyon. Nonparametric Statistics: A Contemporary Approach. Addison-Wesley, 1977.

[128] Stuart Jonathan Russell and Peter Norvig. Artiﬁcial Intelligence: A Modern Approach, chapter 6 Constraint Satisfaction Problems. Prentice Hall, 2010.

[129] Mehrdad Sabetzadeh and Steve Easterbrook. View merging in the presence of incompleteness and inconsistency. Requirements Engineering, 11(3):174–193, 2006.

[130] Mehrdad Sabetzadeh, Davide Falessi, Lionel Briand, and Stefano Di Alesio. A Goal-Based Ap- proach for Qualiﬁcation of New Technologies: Foundations, Tool Support, and Industrial Valida- tion. Reliability Engineering & System Safety, 119(C):52–66, November 2013.

[131] Mazeiar Salehie and Ladan Tahvildari. Self-adaptive Software: Landscape and Research Chal- lenges. ACM Transactions on Autonomous and Adaptive Systems, 4(2):14:1–14:42, May 2009.

[132] Mafalda Santos, Catarina Gralha, Miguel Goulao, Jo˜aoAra´ujo,Ana Moreira, and Joao Cambeiro. What is the Impact of Bad Layout in the Understandability of Social Goal Models? In Proceedings of the 24th IEEE International Requirements Engineering Conference (RE’16), pages 206–215, 2016.

[133] Pete Sawyer, Nelly Bencomo, Danny Hughes, Paul Grace, Heather J. Goldsby, and Betty H. C. Cheng. Visualizing the Analysis of Dynamically Adaptive Systems Using i* and DSLs. In Pro- ceedings of the 2nd International Workshop on Requirements Engineering Visualization (REV’07), pages 3–13, 2007.

[134] Roberto Sebastiani, Paolo Giorgini, and John Mylopoulos. Simple and Minimum-Cost Satisﬁability for Goal Models. In Proceedings of the International Conference on Advanced Information Systems Engineering (CAiSE’04), pages 20–35, 2004.

[135] John Sewell. The Shape of the City: Toronto Struggles with Modern Planning. Heritage Series. University of Toronto Press, 1993.

[136] Forrest Shull, Janice Singer, and Dag I.K. Sjøberg. Guide to Advanced Empirical Software Engi- neering. Springer-Verlag New York, Inc., 2007.

[137] Ian Sommerville. Software Engineering (7th Edition). Pearson Addison Wesley, 2004.

[138] John Sterman. Business Dynamics: Systems Thinking and Modeling for a Complex World. McGraw-Hill, 2000.

[139] Axel van Lamsweerde. Requirements Engineering: From Craft to Discipline. In Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE’08), pages 238–249, 2008. Bibliography 151

[140] Axel van Lamsweerde. Requirements Engineering - From System Goals to UML Models to Software Speciﬁcations. Wiley, 2009.

[141] Mira Vrbaski, Gunter Mussbacher, Dorina Petriu, and Daniel Amyot. Goal Models As Run- time Entities in Context-aware Systems. In Proceedings of the 7th Workshop on [email protected] (MRT’12), pages 3–8, 2012.

[142] Diana L. Webber and Hassan Gomaa. Modeling Variability with the Variation Point Model. In Proceedings of the International Conference on Software Reuse (ICSR’02), pages 109–122, 2002.

[143] Richard White. Planning Toronto: The Planners, The Plans, Their Legacies, 1940-80. UBC Press, 2016.

[144] Claes Wohlin, Per Runeson, Martin Höst,Magnus C Ohlsson, BjörnRegnell, and Anders Wesslén. Experimentation in Software Engineering. Springer Berlin Heidelberg, 2012.

[145] Robert K. Yin. Case Study Research: Design and Methods (3rd Edition). Sage, 2003.

[146] Eric S. Yu. Social Modeling and I*. In Alexander T. Borgida, Vinay K. Chaudhri, Paolo Giorgini, and Eric S. Yu, editors, Conceptual Modeling: Foundations and Applications - Essays in Honor of John Mylopoulos, pages 99–121. Springer-Verlag, Berlin, Heidelberg, 2009.

[147] Eric S. K. Yu. Towards Modeling and Reasoning Support for Early-Phase Requirements Engi- neering. In Proceedings of the 3rd IEEE International Symposium on Requirements Engineering (RE’97), pages 226–235, 1997.

[148] He Zhang and Lin Liu. Modelling Environmental Impact with a Goal and Causal Relation In- tegrated Approach. In Proceedings of the International Congress on International Environmental Modelling and Software (iEMSs’10), 2010. Appendices

152 Appendix A

Waste Managements Example - Analysis Input

The full supplement is available online http: // www. cs. toronto. edu/ ~ amgrubb/ archive/ RE16-Supplement .

GrowingLeaf Input File

Below is the Waste Management example (WME) annotated ﬁle that is generated by the front-end of GrowingLeaf and sent to the back-end. We added java-style comments to explain the syntax.

/* * Example .leaf file with annotations. * All "|" characters represents a tab, and "[]" represents optional elements. */ // Analysis time, max time, and max epoch. // simulationType | maxNumOfSteps | maxNumOfEpochs 11 10 10 // Number of Actors 2 // Lists each actor. // A | ActorID | ActorName | ActorType A a000 City A A a001 Citizens A // Number of intentions. 21 // List of intentional elements (intentions) // I | ActorID | ElementID | ElementType | InitSatValue | ElementName I a000 0000 G 6 Manage City Waste I a000 0001 T 3 Use Current Dump I a000 0002 T 6 Use New Dump I a000 0003 T 6 Build Large Dump I a000 0004 T 6 Build Small Dump

153 Appendix A. Waste Managements Example - Analysis Input 154

I a000 0005 S 6 Positive City Image I a000 0006 S 6 Space in Dump I a000 0007 G 3 Comply with Standards I a000 0008 S 6 Reduce Operating Costs I a000 0009 S 6 Reduce Resident Disturbance I a000 0010 T 6 Process Green Waste I a000 0011 T 6 Build Green Centre I a001 0012 S 6 Willingness to Separate Waste I a001 0013 S 6 Environmental Concern I a001 0014 S 6 Enjoy City I a001 0015 S 6 Quality of Waste Separation I a000 0016 R 6 Purchase Land I a000 0017 G 6 Have Workers Union Contract I a000 0018 T 6 Upgrade Trucks I a000 0019 T 6 Update Truck Route I a000 0020 T 6 GW Education Program // List of links // L | RelationshipA | SrcElementID | TarElementID | [RelationshipB] L OR 0004 0002 L OR 0003 0002 L OR 0001 0000 L OR 0002 0000 L HELPS 0010 0005 L AND 0011 0010 L MAKES 0013 0015 L HELPS 0004 0008 L HELPS 0009 0014 L HELPS 0005 0014 L HURTS 0003 0008 L HURTS 0010 0008 L HURTS 0004 0009 L HELPS 0003 0009 L AND 0018 0010 L MAKES 0012 0015 L DEPENDS 0012 0020 L DEPENDS 0010 0015 // Intention Dynamics // D | ElementID | FunctionType | [InitSatValue] // or // D | ElementID | UD | (begin | End | FuncType | SatValue)* * section represents repeating // or // D | ElementID | UD | (begin | End | FuncType | SatValue)* | repeatBegin | repeatEnd D 0000 NT D 0001 SD D 0002 NT D 0003 RC 3 D 0004 RC 3 Appendix A. Waste Managements Example - Analysis Input 155

D 0005 NT D 0006 MN 0 D 0007 C 3 D 0008 NT D 0009 NT D 0010 NT D 0011 RC 3 D 0012 MP 3 D 0013 R D 0014 NT D 0015 NT D 0016 NT D 0017 CR 3 D 0018 RC 3 D 0019 NT D 0020 NT // Constraints between Intentions // C | constraintType | SrcElementID | srcVariable | TarElementID | tarVariable C = 0011 A 0004 A C = 0001 A 0006 A Appendix B

Controlled Experiment: Eﬀectiveness and Usability

In this appendix, we present study materials used for the controlled experiment presented in Section 6.2. The full supplement, including videos, is available online http: // www. cs. toronto. edu/ ~ amgrubb/ archive/ RE17-Supplement .

B.1 Protocol

On the following pages we present materials used in the experiment discussed in Section 6.2. - Informed Consent Form - Prize Draw Form - Study Publication Notice Form - Study Protocol (i.e., List of Questions asked in Experiment)

156 Appendix B. Controlled Experiment: Effectiveness and Usability 157

Department of Computer Science

Dear Participant:

You’re invited to participate in a research study on the effectiveness of specifying and analyzing goal models. The researchers in this study are Prof. Marsha Chechik and Alicia Grubb, Department of Computer Science.

We are examining the effectiveness of multiple approaches to modeling, evaluating, and analyzing intentions in a goal model. We will be asking you to complete a three-part experiment (including a training component so that you may familiarize yourself with experiment tooling and approaches).

Participating in this study is completely voluntary and you may withdraw from the study at any point during the in-person session, for any reason. If you wish to withdraw, please verbally inform Ms. Grubb and all information collected from you will be destroyed. To maintain confidentiality of your data, it will immediately be anonymized and separated from this form. For this reason, you will not be able to withdraw from the study after data anonymization has taken place. There are no anticipated risks related to this research. The paper copy of this form will be kept in a locked filing cabinet at the University of Toronto. Digital data collected in the study will be stored on a secure password-protected server in the Department of Computer Science and will be encrypted. All information will be destroyed after 5 years.

At the end of the session, you will have the option to be entered into a draw for a number of $50 Amazon gift certificate(s). We will offer 1 gift certificate for every 15 participants, ensuring that you have at least a 1/15 chance of winning. Although you might not benefit directly from this study, your participation will help improve analysis methods within goal modeling. We plan to present the results from this study at academic conferences and in scholarly publications. At no time will your name be used or any identifying information about you revealed. You will also have the option to sign up to receive an update about the study results once published.

If you have any other questions about your rights as a participant in this research, you may also contact the Office of Research Services at the University of Toronto at 416-978-3273 or [email protected].

I have read (or have been read) the above information regarding this research study on the impact of goal modeling approaches and consent to participate in this study.

______(Printed Name in Full)

______(Signature)

______(Date)

Appendix B. Controlled Experiment: Effectiveness and Usability 158

Department of Computer Science

Dear Participant:

Thank you for participating in our study entitled “Investigating the effectiveness of a language for specifying time based satisfaction, and simulation for goal models with dynamism”.

Would you like to be entered into a draw to win a number of $50 Amazon gift certificate(s). We will offer 1 gift certificate for every 15 participants, ensuring that you have at least a 1/15 chance of winning. Participation in this draw is voluntary.

_____ Yes

_____ No

If you answered yes, please provide either an email address or a telephone number so that we may contact you in the event that you are winner.

______

All completed forms will be kept in a locked filing cabinet at the University of Toronto for up to 30 days after the completion of this study.

Appendix B. Controlled Experiment: Effectiveness and Usability 159

Department of Computer Science

Dear Participant:

Thank you for participating in our study entitled “Investigating the effectiveness of a language for specifying time based satisfaction, and simulation for goal models with dynamism”.

Would you like to receive the results of this study once published? Participation is voluntary.

_____ Yes

_____ No

If you answered yes, please provide an email address so that we may contact you with the results of the study once published.

______

All completed forms will be kept in a locked filing cabinet at the University of Toronto for up to 5 years.

Study Protocol Questions (Group A) Questions (Group B) Questions (Group CA) Questions (Group CB) Usability and Effectiveness Experiment: Controlled B. Appendix Part 0 - Informed Consent Participant satisfies eligibility requirements? (Y/N) Signed and reviewed informed consent? (Y/N) On the above scale rate your familiarity with requirements engineering. On the above scale rate your familiarity with the i* (or iStar) modeling language. Randomly Assign Subject to Group Part 1 - Introduction / Training Review or introduce goal modeling and the “Leaf” goal modeling tool (Video 1A - Welcome-Video). Do you have any questions about the video, goal modeling or the leaf tool? Load Model 1 - Trusted Computing Name all the actors in the model. Name all the leaf nodes in the model (nodes without incoming dependencies). Name all the top-level nodes in the model (nodes that do not contribute to other nodes and/or no outgoing dependencies). Introduce Forward Analysis (Video 1B - Forward-Analysis). Do you have any questions about the video or its content? If “Profit” is FS and “Produce PC Products” is FD, what will be the value of “Sell PC Products for Profit”? If “Purchase PC Products” is FS and “Obtain PC Products from Data Pirates” is FD, what will be the value of “PC Products Be Obtained”? If “Allow Peer to Peer Technology” is FS, what will be the value of “Desirable PC Products”? Does “Make Content Available Peer to Peer” propagate its value to “Pirated PC Products” OR does “Pirated PC Products” propagate its value to “Make Content Which is the stronger relationship, “helps” or “depends”? If the "PC Product Provider" can only do one of “Produce PC Products" or "Allow Peer to Peer Technology” which on is best for the top goals (use forward analysis to Part 2 Part 2 Part 2 Part 2 Introduce Dynamic Intention Construct Introduce Stochastic Variation Construct Introduce Advanced Forward Analysis Introduce Advanced Forward Analysis (Video 2A - Dynamics). (Video 2B - Stochastic). (Video 3C - Forward-Analysis- (Video 3C - Forward-Analysis- Advanced). Advanced). Do you have any questions about the Do you have any questions about the Do you have any questions about the Do you have any questions about the video or its content? video or its content? video or its content? video or its content? Load Model 2 - Network Admin Load Model 2 - Network Admin Load Model 3 - City Dump Load Model 3 - City Dump If “Develop Project” has M+ as its Identify which elements in this model Identify all the alternative decisions in Identify all the alternative decisions in dynamic type and PD as its current change over time? the model. the model. evaluation, what possible evaluations will it have in the future? If “Political Will” has R as its dynamic How do each of these elements change? Use repeated forward analysis (varying Use repeated forward analysis (varying type and PD as its current evaluation, initial evaluations) to choose the best initial evaluations) to choose the best what possible evaluations will it have in alternative for each decision. alternative for each decision. the future? Identify which elements in this model If we consider the model to be In your own words describe how the In your own words describe how the change over time? stochastic and “Political Will” has PD as element evaluations vary over your element evaluations vary over your its current evaluation, what possible analysis? Do any trends emerge? analysis? Do any trends emerge? evaluations will it have in the future? How do each of these elements change? Are there any nodes in the model that For each alternative decision, which goal For each alternative decision, which goal you could describe better than has the greatest impact on the has the greatest impact on the stochastic? If so, which ones and how. decision? decision? 160 Which elements (if any) would you have For the purpose of communicating with What would be the impact if What would be the impact if assigned a different dynamic function? stakeholders, how would you represent “Environmental Concern” changes in the “Environmental Concern” changes in the Usability and Effectiveness Experiment: Controlled B. Appendix these dynamics symbolically in the future? future? model? If you can only do one of “Update If you can only do one of “Update Assume you can sequentially complete Assume you can sequentially complete Current Technology” or “Increase Current Technology” or “Increase both “Build Green Centre” and “Build both “Build Green Centre” and “Build Capacity” which one is best for the top Capacity” which one is best for the top Small Dump”. Which order is best for Small Dump”. Which order is best for goals (use forward analysis to evaluate goals (use forward analysis to evaluate the top goals (use forward analysis to the top goals (use forward analysis to the alternatives)? Why? the alternatives)? Why? evaluate the alternatives)? Why? evaluate the alternatives)? Why? If you had drawn this model, would you If you had drawn this model, would you If you had drawn this model, would you If you had drawn this model, would you have drawn it differently? If so, how? have drawn it differently? If so, how? have drawn it differently? If so, how? have drawn it differently? If so, how?

Part 3 Part 3 Part 3 Part 3 Introduce Simulation with Dynamic Introduce Simulation with Stochastic Introduce Dynamic Intention Construct Introduce Stochastic Variation Construct Intentions (Video 3A - Dynamic- Intentions (Video 3B - Stochastic- (Video 2A - Dynamics). (Video 2B - Stochastic). Analysis). Analysis). Do you have any questions about the Do you have any questions about the Do you have any questions about the Do you have any questions about the video or its content? video or its content? video or its content? video or its content? Load Model 3 - City Dump Load Model 3 - City Dump Load Model 2 - Network Admin Load Model 2 - Network Admin Identify all the alternative decisions in Identify all the alternative decisions in If “Develop Project” has M+ as its Identify which elements in this model the model. the model. dynamic type and PD as its current change over time? evaluation, what possible evaluations will it have in the future? Use the simulator to choose the best Use the simulator to choose the best If “Political Will” has R as its dynamic How do each of these elements change? alternative for each decision. alternative for each decision. type and PD as its current evaluation, what possible evaluations will it have in the future? In your own words describe how the In your own words describe how the Identify which elements in this model If we consider the model to be element evaluations vary over the element evaluations vary over the change over time? stochastic and “Political Will” has PD as simulation? Do any trends emerge? simulation? Do any trends emerge? its current evaluation, what possible evaluations will it have in the future? For each alternative decision, which goal For each alternative decision, which goal How do each of these elements change? Are there any nodes in the model that has the greatest impact on the has the greatest impact on the you could describe better than decision? decision? stochastic? If so, which ones and how. What would be the impact if What would be the impact if Which elements (if any) would you have For the purpose of communicating with “Environmental Concern” changes in the “Environmental Concern” changes in the assigned a different dynamic function? stakeholders, how would you represent future? future? these dynamics symbolically in the model? Assume you can sequentially complete Assume you can sequentially complete If you can only do one of “Update If you can only do one of “Update both “Build Green Centre” and “Build both “Build Green Centre” and “Build Current Technology” or “Increase Current Technology” or “Increase Small Dump”. Which order is best for Small Dump”. Which order is best for Capacity” which one is best for the top Capacity” which one is best for the top the top goals (use simulation to the top goals (use simulation to goals (use forward analysis to evaluate goals (use forward analysis to evaluate evaluate the alternatives)? Why? evaluate the alternatives)? Why? the alternatives)? Why? the alternatives)? Why? If you had drawn this model, would you If you had drawn this model, would you If you had drawn this model, would you If you had drawn this model, would you have drawn it differently? If so, how? have drawn it differently? If so, how? have drawn it differently? If so, how? have drawn it differently? If so, how?

Part 4 - Debriefing

What suggestions or changes would you recommend to the developers of this goal modeling tool? 161 Scale: 1 – Completely Dissatisfied, 2 – Mostly Dissatisfied, 3 – Somewhat Dissatisfied, 4 – Neither Satisfied or Dissatisfied, 5 – Somewhat Satisfied, 6 – Mostly Satisfied, 7 – Completely satisfied Usability and Effectiveness Experiment: Controlled B. Appendix On a scale above rate your level of satisfaction with the tools: ease of use appearance modeling functionality analysis functionality Scale: 1 – Extremely Unlikely, 2 – Unlikely, 3 – Neutral, 4 – Likely, 5 – Extremely Likely How likely is it that you would recommend this goal modeling tool to a colleague? What is your current program of study? (Bachelors, Masters, PhD, Non-Student) Participant Goal Modeling Feedback and Discussion: The participant will be given feedback by the researcher on their performance in basic goal modeling. Study Update Form and Thank You 162 Appendix B. Controlled Experiment: Effectiveness and Usability 163

B.2 Recruitment Materials

The following message was used to recruit subjects from the Faculty of Information. Is was posted to the INF1341 course message board.

Hi all,

As a student in INF1341, I would like to invite you to participate in a study examining the eﬀectiveness of multiple approaches to modeling, evaluating, and analyzing intentions in a goal model. INF1341 has given you the relevant expertise to participate in our study. We hope to gain insights on how to best design tools for software modelers and stakeholders. Hence, we?re looking for participants who have a basic understanding of requirements engineering and are proﬁcient in English (both written and verbal).

If you ﬁt these criteria, this is your chance to learn more about goal modeling. You will be given one-on-one feedback to help improve your own modeling skills. In addition, upon completion of the study, you have the option to be entered into a draw for a number of $50 Amazon gift certiﬁcate(s). The study is expected to take approximately 60 minutes.

If you know people who ﬁt the criteria and might be willing to participate, please forward this information with the attachment and ask them to contact me.

The informed consent letter is available at http: // www. cs. toronto. edu/ ~ amgrubb/ study. pdf . The letter informs you of the purpose of the study, and the risks and rewards associated with participating. Please review it carefully to help you determine if you wish to participate.

If you would like to participate or have any questions, please email me at [email protected].

Thanks! Alicia Grubb Appendix B. Controlled Experiment: Effectiveness and Usability 164

The following message was used to recruit subjects from the Faculty of Arts and Science. It was sent to the software engineering research mailing list.

Hi all,

What is requirements engineering?

If you can answer this question, I would like to invite you to participate in a study examining the eﬀectiveness of multiple approaches to modeling, evaluating, and analyzing intentions in a goal model. We hope to gain insights on how to best design tools for software modelers and stakeholders. Hence, we?re looking for graduate student participants who have a basic understanding of requirements engineering and are proﬁcient in English (both written and verbal).

If you ﬁt these criteria, this is your chance to learn more about goal modeling. Par- ticipants will be given one-on-one feedback to help improve their own modeling skills. In addition, upon completion of the study, you have the option to be entered into a draw for a number of $50 Amazon gift certiﬁcate(s). The study is expected to take approximately 60 minutes.

If you know people who ﬁt the criteria and might be willing to participate, please forward this email and ask them to contact me.

If you would like to participate or have any questions, please email me at am- [email protected].

Thanks! Alicia Grubb Appendix B. Controlled Experiment: Effectiveness and Usability 165

B.3 Video Scripts

In this section, we present the transcripts from the videos used in Section 6.2. The videos are available online http: // www. cs. toronto. edu/ ~ amgrubb/ archive/ RE17-Supplement .

Video 1A

Welcome to Growing Leaf, a tool for modeling and analyzing iStar goal models. The tool consists of a stencil on the left, a canvas in the centre, and the property view on the right. The centre canvas has a model of the Trusted Computing Case Study. Let’s start by introducing this model as you will use it in the next step. Relevant actors in this situation include the PC user, PC product producer and the data pirate. A PC user can acquire PC products legally or acquire them illegally through a data pirate. When acquiring PC products, a PC user may be interested in both saving money, and in abiding by licensing regulations. Legal acquisition helps to abide by licensing regulations, but has a negative effect on the software users desire to find affordable products. Acquiring PC products illegally helps to satisfy the goal of finding affordable products, but breaks licensing regulations. From a different perspective, the PC product producer wants to sell PC products in order to make a profit. In order to be profitable, the PC product provider must make PC products desirable and also depends on the PC user to abide by licensing regulations. Allowing the use of peer-to-peer technology in PC products can help to make these products more desirable, but can also allow the Data Pirate the ability to make pirated content available. To add an element to the canvas from the stencil simply drag it on to the canvas. To rename any element or change the satisfaction value, first click on the element, and then update the name in the Node Name field or change the Initial Satisfaction Value in the drop-down menu. When editing the name of an element, do not use the enter key, but instead use the mouse to click out of the Node Name box. Here we added another Actor called PC Corporate User, a Goal PC Products Be Obtained, and a task Purchase PC Products. We can now create links. In order to create a link, click on the element source. Here we have click on Purchase PC Products. Notice three icons have appeared above the element. From left to right they are: resize, delete, and create a link. We click and drag the link icon to the destination goal PC Products Be Obtained. By default an AND-Decomposition link is created. You can change the link type by hovering the link until the gear and delete icon appears and then clicking the gear icon. This populates the relation type in the property view on the right. The drop down menu allows you to select a different link type. Here we have selected OR Decomposition which is also known as Means End. You can move the canvas or any element on it around by clicking on it and dragging to its desired location. All other features are located in the top Toolbar. You can Zoom in and out, and change the font size. At various points in this study you will be asked to load a model by clicking the Load icon, so we point it out now. Do not use the delete keyboard key ever. Now we need to introduce a bit of terminology. Model nodes, that is Goals, Tasks, Resources, and Soft Goals will form a graph. Leaf Nodes are defined as nodes with NO Incoming decomposition (AND/OR [Means-End]) or contribution links (Makes, Breaks, Helps, Hurts) and NO Outgoing dependency links (Depends). Appendix B. Controlled Experiment: Effectiveness and Usability 166

Root Nodes or Top-Level Nodes are deﬁned as nodes with NO Outgoing decomposition (AND/OR [Means-End]) or contribution links (Makes, Breaks, Helps, Hurts) and NO Incoming dependency links (Depends). In this example, if we look only at the PC Corporate User actor, PC Products Be Obtained is a Root Node or Top-Level Node. and Purchase PC Products is a Leaf Node. You can also review these deﬁnitions in the Study Terminology and Reference provided. If you have any further questions please ask. You can also refer to the provided iStar cheat sheet. This concludes this video.

Video 1B

This video introduces the analysis view for GrowingLeaf. All of the analysis you will do today is based on one technique called forward analysis. Forward Analysis propagates values from Leaf Nodes to Root Nodes or Top-Level Nodes, following the propagation rules of each link type. Recall from the last video that: Leaf Nodes are defined as nodes with NO Incoming decomposition or contribution links, and NO Outgoing dependency links. and Root Nodes or Top-Level Nodes are defined as nodes with NO Outgoing decomposition or contribution links, and NO Incoming dependency links. In the partial model at the bottom, Goal 83, Task 89, and Task 92 are Leaf Nodes and have been given initial values. Goal 81 and Soft Goal 84 are Root Nodes. This collection of nodes and links will show you how Forward Analysis works. Dependency links propagate the value at the target to the value at the source. For example Goal 15 propagates the Satisfied label to Goal 14. Decomposition: When a goal (or task) has an AND-Decomposition or OR-Decomposition (Means- End) with one or more children, we can calculate the value of the goal using the ordering: Fully Denied < Partially Denied < Conflict < Unknown < Partially Satisfied < Fully Satisfied. For AND-Decomposition we take the minimum value of all the child nodes. For OR-Decomposition we take the maximum value of all the child nodes. See four examples of decomposition at the top. Contribution Links: Contribution links propagate based on the type of link. This example contains all the permutations of the link type with the initial evaluation labels: Fully Denied, Partially Denied, Partially Satisfied, and Satisfied. You can also refer to the provided Study Terminology and Reference sheet, for the full table of contribution and decomposition links. With any model, once you have assigned all the initial evaluation labels to the Leaf Nodes you can run forward analysis. To run forward analysis, click the green Analysis in the top Tool Bar and the Analysis View will show. Then click Forward Analysis from the Right Panel. The analysis will compute and will show up in the History Log on the left panel. The model now contains the results of the analysis. Notice a slider has appeared on the bottom of the screen that allows you to click between the original and analyzed model. The model at the bottom shows you that forward analysis continues to propagate over intermediate nodes, and propagates values throughout the model. The purpose of forward analysis is to ask what if questions of the model. By varying the evaluations of Leaf Nodes you can determine their impact on Top-Level Nodes. This is a way of testing the impact of possible scenarios. Appendix B. Controlled Experiment: Effectiveness and Usability 167

You can the video now to review the results of forward analysis. To navigate back to the Modeling view, move the indicator on the slider back to the initial position on the left, and then click Model then click Model Current Analysis. Note: If you do not move the slider back to the initial position, the current evaluation labels will show in the Modeling View. If you have any further questions please ask. This concludes this video.

Video 2 (2A & 2B)

In this video we explore what it means for goal model elements to change over time. Next we look at a model for a Network Administrator at an Internet Service Provider. Their Top- Level goals are to Improve Network Infrastructure and to Increase Customers. They can Increase Cus- tomers by having suﬃcient capacity on the network and by having a reliable network. The trade-oﬀs the Network Admin is considering making is whether to Maintain Network, Update Current Technology, and Increase Capacity on the network. Increasing capacity is decomposed into getting capital funding and developing the project.

Video 2A - Insert

We extend i* to enable analysis where the Leaf Nodes in the model change over time. Recall to evaluate goal models we use the evaluation labels: Fully Denied, Partially Denied, Partially Satisfied, and Fully Satisfied. Note: Unknown and Conflict labels are not used for Leaf Nodes in analysis. If the Leaf Nodes change Stochastically over time, this means that in the next state they can have any of the evaluation labels including the same one as the previous state. When we look at any interval, we can say the value increases, decreases, remains constant, or is stochastic, and have named elementary functions for these. We also have functions that define the behaviour over multiple intervals. Denied-Satisfied says that the value will be Fully Denied until a time n when the value will be Fully Satisfied. Similarly, Satisfied- Denied says the value will be Fully Satisfied until a time n when the value will be Fully Denied. Monotonic Negative says the value will decrease until a time n when it will remain Fully Denied. Monotonic Positive says the value will increase until a time n when it will remain Fully Satisfied. Here is a list of all our component functions. We have provided this list on the reference sheet so that you can review them at your own pace. This list is incomplete and you can create your own functions if you wish. Going back to the model, if we assign some initial dynamics, here is what a projection of this model might look like over time. Notice that the bottom left corner of some Leaf Nodes have a letter to indicate what dynamic function is used for this analysis. If you have any further questions please ask. This concludes this video.

Video 2B - Insert

If the Leaf Nodes change Stochastically over time, this means that in the next state they can have any of the evaluation labels including the same one as the previous state. With our Network Administrator model we can use repeated forward analysis to understand the impacts of Stochastic Leaf Nodes. Here we iterate through a collection of results, and can imagine the values changing over time. If you have any further questions please ask. This concludes this video.

Video 3A/3B

In this video we explore how to analyze models over time. The next model you will look at is of a city evaluating its waste management infrastructure. The city currently has a landﬁll (dump) that has not reached capacity and is considering investing in building a new one as well as a recycling and composting facility. This model shows the City and its Citizens. The city wants to satisfy Manage City Waste, Comply with Standards, Reduce Operating Costs, Have Workers Union Contract, and for their citizens to Enjoy City, their Top-level goals.

Video 3A - Insert

In the last video, we introduced dynamic functions that you can assign to Leaf Nodes, now we show you how to simulate these functions over time. First assign initial evaluation labels and dynamic types to your leaf nodes. To do this click on each node and select the values from the properties panel on the right. The dynamic functions are now listed in the drop down Function Type menu. Once you have set up your model, navigate to the analysis view (by clicking Analysis from the top toolbar). Then click Leaf Simulate from the right panel. The analysis will compute and will show up in the History Log on the left panel. The model now contains the results of the simulation. Notice the slider at the bottom on the screen now has 20 simulation steps. You can scroll through the simulation to see how the model evaluations have changed as a result of the dynamic functions. Remember, when you click Model and then Model From Current State, the values based on the current slider position will be shown in the modeling view. You can repeat this process, changing the dynamic function types and initial values, to understand trends in the model. If you have any further questions please ask. This concludes this video.

Video 3B - Insert

In the last video, we introduced stochastically changing Leaf Nodes, now we show you how to use stochastic simulation to change the values over time. First assign initial evaluation labels to your leaf nodes. To do this click on each node and select the values from the properties panel on the right. Once you have set up your model, navigate to the analysis view (by clicking Analysis from the top toolbar). Then click Stochastic Simulation from the right panel. The analysis will compute and will show up in the History Log on the left panel. The model now contains the results of the simulation. Appendix B. Controlled Experiment: Effectiveness and Usability 169

Notice the slider at the bottom on the screen now has 20 simulation steps. You can scroll through the simulation to see how the model evaluations have changed as a result of the stochastic values. Remember, when you click Model and then Model From Current State, the values based on the current slider position will be shown in the modeling view. You can repeat this process to understand trends in the model. If you have any further questions please ask. This concludes this video.

Video 3C

In this video we explore an advanced way to use forward analysis. The next model you will look at is of a city evaluating its waste management infrastructure. The city currently has a landﬁll (dump) that has not reached capacity and is considering investing in building a new one as well as a recycling and composting facility. This model shows the City and its Citizens. The city wants to satisfy Manage City Waste, Comply with Standards, Reduce Operating Costs, Have Workers Union Contract, and for their citizens to Enjoy City, their Top-level goals. Now that you know how to use forward analysis, you can use it to understand model alternatives and consider what would happen if instead of choosing alternatives all at once, you can choose them one at a time. You can also choose to satisfy tasks in a sequence. To do this, assign initial satisfaction labels to the model and navigate to the analysis view (by clicking Analysis from the top toolbar). Click Forward Analysis and values will be propagated through out the model. If you like these values, leave the slider in the right most position and click Model and then Model From Current State. Back at the modeling view, you can update one or more of the initial satisfaction values and return to the analysis view to run Forward Analysis again with the new values. You can repeat this process to understand trends in the model. If you have any further questions please ask. This concludes this video. Appendix C

CSP Encoding

C.1 CSP Background

In this section, we describe how we encode a graph evaluation path (see Deﬁnition 4.5) as a constraint satisfaction problem (CSP) [128], allowing us to answer questions using strategy ASR2 (see Section 4.3.1) without the need for our brute force algorithm (see Algorithm 4.3).

Deﬁnition C.1 (Constraint Satisfaction Problem (CSP)). A constraint satisfaction problem is deﬁned as a triple hV , D, Ci, where V = {v1, ..., vn} is a set of variables, D = {d1, ..., dn} is a set of the respective domains of values, and Q = {q1, ..., qm} is a set of constraints [128].

Recall that a complete graph evaluation path is the complete evaluation of a goal graph M = hA, G, R, EF , MC, maxTimei at each time point in Π, given UEval and CFLevel (see Definition 4.5). When constructed as a CSP, our goal is to find an evaluation for each intention in the graph (g ∈ G) at each time point (t ∈ Π). Thus, the CSP variables V are a set of intentions at all time points (i.e., |V | = |G|∗|Π|). Initially, the domain d of each variable v is the set of non-conflicting values (E\CFLevel). Each user evaluation hg, t, ei (see Definition 4.2) reduces the domain for the variable hg, ti to a domain of e. Each element in R and EF forms a constraint q over the elements in V . Once all the constraints are added to the problem, we invoke the JaCoP constraint solver [88] to find satisfying assignments to each variable, such that each variable assignment is an element of the variable’s domain and all the assignments taken together satisfy all constraints in the problem [128]. In this section, we walk through the encoding of a path over an evolving goal graph into a constraint solving problem (CSP). First, we create the structures over which we add the constraints. We introduce constraints in the following order: relationships (and evolving relationships), evolving functions (i.e., atomic and User-Defined functions), NotBoth evolving function, constraints between time points, and finally conflict avoidance and user evaluations.

170 Appendix C. CSP Encoding 171

C.2 Time Point and Goal Evaluations

The goal in computing a graph evaluation path is to assign values for each time point and each goal in the graph at each time point. First, we define the data structures for time points and goal evaluations. Initialize Time Points takes as input a set of graph time points TP (M) (see Definition 4.1) and creates an Integer Array tV al. It also ensures that each selected time will be less than maxT ime. If the CSP solver finds a solution, tV al will be a time point path satisfying Definition 4.4. When we indicate CSP .add{...} as in Initialize Time Points, this specifies the details of a constraint added to the CSP solver. 1: function Initialize Time Points(TP (M), maxT ime) 2: tV al = newInteger[|TP (M)|] 3: for all p ∈ TP (M) do 4: if p.index = 0 then 5: CSP .add{ 6: tV al[0] = 0 7: } 8: else if p 6= ⊥ then 9: CSP .add{ 10: tV al[p.index] = p J K 11: } 12: else 13: CSP .add{ 14: 0 ≤ tV al[p.index] ≤ maxT ime 15: } 16: return tV al

Encoding the Bike Lanes Example. Throughout this section, we illustrate how pieces of the Bike Lanes example would be encoded into CSP. In our presentation of the constraints over the Bike Lanes example, we take liberty with array notation to make it readable.

We ﬁrst create the data structures over which to add constraints. Calling Initialize Time Points with the graph time points from the speciﬁcation in Figure 3.5 creates 22 time points tV al[0–21], one for each element in the Time Points column in Table 4.2. It assigns values to each of the time points, and ranges to the rest.

tV al[0] = 0, tV al[tuse] = 10, tV al[tBLOpen] = 10,

tV al[tdecision] = 5, tV al[ta0] = 7, tV al[ta1 ] = 14, and tV al[ta2 ] = 34 for the absolute values.

tV al[tdesign] > 0 ∧ tV al[tdesign] ≤ maxT ime,

tV al[r1(tspring)] > 0 ∧ tV al[r1(tspring)] ≤ maxT ime, ···

tV al[r3(tfall)] > 0 ∧ tV al[r3(tfall)] ≤ maxT ime. Appendix C. CSP Encoding 172

Initialize Goals takes as input the created time point array tV al, and the set of goals in the model G. Initialize Goals creates a boolean array to hold the four evidence predicates for each goal at each time point. It then adds the ground axioms from Section 2.2 (see lines 3–10). Finally, Initialize Goals adds the constraint to all goals that if two time points have equivalent values then the evaluations of all goals at those two time points must be the same (see lines 11–18). For ease of readability, we vary the notation slightly. We write gV al[g][t][FD] =⇒ gV al[g][t][PD] instead of if gV al[g][t][FD] = true then gV al[g][t][PD] = true and use gV al[g][t] to denote the assignment of an evidence pair, instead of having four assignments (one for each evidence predicate).

1: function Initialize Goals(G, tV al) 2: gV al = newBoolean[|G|][|tV al|][4] 3: for all t ∈ tV al do 4: for all g ∈ gV al do 5: CSP .add{ 6: gV al[g][t][FD] =⇒ gV al[g][t][PD] 7: gV al[g][t][FS] =⇒ gV al[g][t][PS] 8: }

9: for all t1 ∈ tV al do

10: for all t2 ∈ tV al do 11: for all g ∈ gV al do 12: CSP .add{

13: (t1 = t2) =⇒ gV al[g][t1] = gV al[g][t2] 14: } 15: return gV al Appendix C. CSP Encoding 173

Next, calling Initialize Goals with tV al and the graph in Figure 3.5, creates the array gV al[8][22][4], where each element of the array is a single predicate (FD, PD, PS, FS) at a single time point for a single goal. Then two sets of constraints are added with the results listed below. The ground axioms from Section 2.2 are added and then for every combination of two time points and a goal, we add the constraint that if the evaluation of any two time points is the same, the evaluation of the goal at each time point is the same as well. Now that we have created the data structures required, we can add the constraints over them.

gV al[Bike Lane Usage][t0][FD] =⇒ gV al[Bike Lane Usage][t0][PD]

gV al[Bike Lane Usage][t0][FS] =⇒ gV al[Bike Lane Usage][t0][PS]

gV al[Bike Lane Usage][tdesign][FD] =⇒ gV al[Bike Lane Usage][tdesign][PD]

gV al[Bike Lane Usage][tdesign][FS] =⇒ gV al[Bike Lane Usage][tdesign][PS] ···

gV al[Have Bike Lanes][r3(tsummer)][FD] =⇒ gV al[Have Bike Lanes][r3(tsummer)][PD]

gV al[Have Bike Lanes][r3(tsummer)][FS] =⇒ gV al[Have Bike Lanes][r3(tsummer)][PS]

gV al[Have Bike Lanes][r3(tfall)][FD] =⇒ gV al[Have Bike Lanes][r3(tfall)][PD]

gV al[Have Bike Lanes][r3(tfall)][FS] =⇒ gV al[Have Bike Lanes][r3(tfall)][PS]

(tV al[0] = tV al[1]) =⇒ gV al[Bike Lane Usage][0] = gV al[Bike Lane Usage][1] (tV al[0] = tV al[2]) =⇒ gV al[Bike Lane Usage][0] = gV al[Bike Lane Usage][2] ··· (tV al[i] = tV al[j]) =⇒ gV al[g][i] = gV al[g][j] ··· (tV al[20] = tV al[21]) =⇒ gV al[Have Bike Lanes][20] = gV al[Have Bike Lanes][21]

C.3 Relationships & Evolving Relationships

We add constraints for forward propagation rules (see Figure 2.8(a)) and backwards propagation rules (see Figure 2.8(b)) separately. The Link Constraints procedure starts by iterating over all relationships to add the forward propagation rules, separating out NotBoth relationships (lines 3–4) and evolving relationship (lines 5–6). If it is not an evolving relationship, Forward is called to add the constraints for the individual relationship. The procedure ﬁnishes by calling the Backward procedure which adds the constraints for the backwards rules. Appendix C. CSP Encoding 174

1: procedure Link Constraints(G, R, tV al, gV al) 2: for all r ∈ R do . Forward Propagation 3: if r.type = NotBoth then 4: Not Both Constraint(tV al, gV al, r) 5: else 6: g = r.targetGoal 7: sGoals = r.sourceGoals 8: if r.isEvolving 6= true then 9: type = r.type 10: for all t ∈ tV al do 11: Forward(g, sGoals, type, t, gV al) 12: else 13: ... . Omitted to avoid duplication. Each half of the evolving relationship is handles separately. 14: Backward(G, R, tV al, gV al)

Forward Propagation

1: procedure Forward(g, sGoals, type, t, gV al) 2: if type = and then . And Relationship 3: F SV al is a gV al array size of sGoals 4: P SV al is a gV al array size of sGoals 5: for all s ∈ sGoals do 6: F SV al.add(gV al[s][t][FS]) 7: P SV al.add(gV al[s][t][PS]) 8: CSP .add{ 9: (CSP .AND F SV al) =⇒ gV al[g][t][FS] 10: (CSP .AND P SV al) =⇒ gV al[g][t][PS] 11: } 12: for all s ∈ sGoals do 13: CSP .add{ 14: gV al[s][t][FD] =⇒ gV al[g][t][FD] 15: gV al[s][t][PD] =⇒ gV al[g][t][PD] 16: } 17: else if type = or then . Or Relationship 18: F DV al is a gV al array size of sGoals 19: P DV al is a gV al array size of sGoals 20: for all s ∈ sGoals do 21: F DV al.add(gV al[s][t][FD]) 22: P DV al.add(gV al[s][t][PD]) 23: CSP .add{ 24: (CSP .AND F DV al) =⇒ gV al[g][t][FD] 25: (CSP .AND P DV al) =⇒ gV al[g][t][PD] 26: } 27: for all s ∈ sGoals do 28: CSP .add{ 29: gV al[s][t][FS] =⇒ gV al[g][t][FS] Appendix C. CSP Encoding 175

30: gV al[s][t][PS] =⇒ gV al[g][t][PS] 31: } 32: else . Contribution Relationships

33: g1 = sGoals[0] 34: if type = +S then 35: CSP .add{

36: gV al[g1][t][PS] =⇒ gV al[g][t][PS] 37: } 38: else if type = -S then 39: CSP .add{

40: gV al[g1][t][PS] =⇒ gV al[g][t][PD] 41: } 42: else if type = + + S then 43: CSP .add{

44: gV al[g1][t][FS] =⇒ gV al[g][t][FS]

45: gV al[g1][t][PS] =⇒ gV al[g][t][PS] 46: } 47: else if type = --S then 48: CSP .add{

49: gV al[g1][t][FS] =⇒ gV al[g][t][FD]

50: gV al[g1][t][PS] =⇒ gV al[g][t][PD] 51: } 52: else if type = +D then 53: CSP .add{

54: gV al[g1][t][PD] =⇒ gV al[g][t][PD] 55: } 56: else if type = -D then 57: CSP .add{

58: gV al[g1][t][PD] =⇒ gV al[g][t][PS] 59: } 60: else if type = + + D then 61: CSP .add{

62: gV al[g1][t][FD] =⇒ gV al[g][t][FD]

63: gV al[g1][t][PD] =⇒ gV al[g][t][PD] 64: } 65: else if type = --D then 66: CSP .add{

67: gV al[g1][t][FD] =⇒ gV al[g][t][FS]

68: gV al[g1][t][PD] =⇒ gV al[g][t][PS] 69: } 70: else if type = + then 71: CSP .add{

72: gV al[g1][t][PS] =⇒ gV al[g][t][PS]

73: gV al[g1][t][PD] =⇒ gV al[g][t][PD] 74: } 75: else if type = - then 76: CSP .add{ Appendix C. CSP Encoding 176

77: gV al[g1][t][PS] =⇒ gV al[g][t][PD]

78: gV al[g1][t][PD] =⇒ gV al[g][t][PS] 79: } 80: else if type = ++ then 81: CSP .add{

82: gV al[g1][t][FS] =⇒ gV al[g][t][FS]

83: gV al[g1][t][PS] =⇒ gV al[g][t][PS]

84: gV al[g1][t][FD] =⇒ gV al[g][t][FD]

85: gV al[g1][t][PD] =⇒ gV al[g][t][PD] 86: } 87: else if type = -- then 88: CSP .add{

89: gV al[g1][t][FS] =⇒ gV al[g][t][FD]

90: gV al[g1][t][PS] =⇒ gV al[g][t][PD]

91: gV al[g1][t][FD] =⇒ gV al[g][t][FS]

92: gV al[g1][t][PD] =⇒ gV al[g][t][PS] 93: }

Backward Propagation

1: procedure Backward(G, R, tV al, gV al) 2: for all g ∈ G do 3: for all t ∈ tV al do 4: F SCon = {}, P SCon = {} 5: P DCon = {}, F DCon = {} 6: rSet = R.getT argetRel(g, t) . Gets the set of relationships that have g as the target goal at time t. Accounts for evolving relationships. 7: for all r ∈ rSet do 8: type = r.type 9: sGoals = r.sourceGoalSet

10: g1 = sGoals[0] 11: if type = and ∨ type = or then 12: sV alue = P rimitiveConstraint[4][|sGoals|] . Holds CSP constraints. 13: for all s ∈ sGoals do 14: sV alue[FS][s] = gV al[s][t][FS] 15: sV alue[PS][s] = gV al[s][t][PS] 16: sV alue[PD][s] = gV al[s][t][PD] 17: sV alue[FD][s] = gV al[s][t][FD] 18: if type = and then 19: F SCon.add(And(sV alue[FS])) 20: P SCon.add(And(sV alue[PS])) 21: P DCon.add(Or(sV alue[PD])) 22: F DCon.add(Or(sV alue[FD])) 23: else 24: F SCon.add(Or(sV alue[FS])) 25: P SCon.add(Or(sV alue[PS])) 26: P DCon.add(And(sV alue[PD])) 27: F DCon.add(And(sV alue[FD])) Appendix C. CSP Encoding 177

28: else if type = +S then

29: P SCon.add(gV al[g1][t][PS]) 30: else if type = -S then

31: P DCon.add(gV al[g1][t][PS]) 32: else if type = + + S then

33: F SCon.add(gV al[g1][t][FS])

34: P SCon.add(gV al[g1][t][PS]) 35: else if type = --S then

36: P DCon.add(gV al[g1][t][PS])

37: F DCon.add(gV al[g1][t][FS]) 38: else if type = +D then

39: P DCon.add(gV al[g1][t][PD]) 40: else if type = -D then

41: P SCon.add(gV al[g1][t][PD]) 42: else if type = + + D then

43: P DCon.add(gV al[g1][t][PD])

44: F DCon.add(gV al[g1][t][FD]) 45: else if type = --D then

46: F SCon.add(gV al[g1][t][FD])

47: P SCon.add(gV al[g1][t][PD]) 48: else if type = + then

49: P SCon.add(gV al[g1][t][PS])

50: P DCon.add(gV al[g1][t][PD]) 51: else if type = - then

52: P SCon.add(gV al[g1][t][PD])

53: P DCon.add(gV al[g1][t][PS]) 54: else if type = ++ then

55: F SCon.add(gV al[g1][t][FS])

56: P SCon.add(gV al[g1][t][PS])

57: P DCon.add(gV al[g1][t][PD])

58: F DCon.add(gV al[g1][t][FD]) 59: else if type = -- then

60: F SCon.add(gV al[g1][t][FD])

61: P SCon.add(gV al[g1][t][PD])

62: P DCon.add(gV al[g1][t][PS])

63: F DCon.add(gV al[g1][t][FS]) 64: if |F SCon| > 0 then . Apply constraints. 65: CSP .add{ 66: if gV al[g][t][FS] then Or(F SCon) 67: } 68: if |P SCon| > 0 then 69: CSP .add{ 70: if gV al[g][t][PS] then Or(P SCon) 71: } 72: if |P DCon| > 0 then Appendix C. CSP Encoding 178

73: CSP .add{ 74: if gV al[g][t][PD] then Or(P DCon) 75: } 76: if |F DCon| > 0 then 77: CSP .add{ 78: if gV al[g][t][FD] then Or(F DCon) 79: }

Calling Link Constraints creates constraints for each time point and each relationship in the graph in Figure 3.5. We illustrate the forward analysis constraints for Have Bike Lanes below.

and // (Have Design, Build Bike Lanes) −−−→ Have Bike Lanes gV al[Have Design][0][FS] ∧ gV al[Build Bike Lanes][0][FS] =⇒ gV al[Have Bike Lanes][0][FS] gV al[Have Design][0][PS] ∧ gV al[Build Bike Lanes][0][PS] =⇒ gV al[Have Bike Lanes][0][PS] gV al[Have Design][1][FS] ∧ gV al[Build Bike Lanes][1][FS] =⇒ gV al[Have Bike Lanes][1][FS] gV al[Have Design][1][PS] ∧ gV al[Build Bike Lanes][1][PS] =⇒ gV al[Have Bike Lanes][1][PS] ··· gV al[Have Design][21][FS] ∧ gV al[Build Bike Lanes][21][FS] =⇒ gV al[Have Bike Lanes][21][FS] gV al[Have Design][21][PS] ∧ gV al[Build Bike Lanes][21][PS] =⇒ gV al[Have Bike Lanes][21][PS]

gV al[Have Design][0][FD] =⇒ gV al[Have Bike Lanes][0][FD] gV al[Have Design][0][PD] =⇒ gV al[Have Bike Lanes][0][PD] gV al[Build Bike Lanes][0][FD] =⇒ gV al[Have Bike Lanes][0][FD] gV al[Build Bike Lanes][0][PD] =⇒ gV al[Have Bike Lanes][0][PD] gV al[Have Design][1][FD] =⇒ gV al[Have Bike Lanes][1][FD] gV al[Have Design][1][PD] =⇒ gV al[Have Bike Lanes][1][PD] gV al[Build Bike Lanes][1][FD] =⇒ gV al[Have Bike Lanes][1][FD] gV al[Build Bike Lanes][1][PD] =⇒ gV al[Have Bike Lanes][1][PD] ··· gV al[Have Design][21][FD] =⇒ gV al[Have Bike Lanes][21][FD] gV al[Have Design][21][PD] =⇒ gV al[Have Bike Lanes][21][PD] gV al[Build Bike Lanes][21][FD] =⇒ gV al[Have Bike Lanes][21][FD] gV al[Build Bike Lanes][21][PD] =⇒ gV al[Have Bike Lanes][21][PD]

++S // Have Bike Lanes −−−→ Cyclist Safety gV al[Have Bike Lanes][0][FS] =⇒ gV al[Cyclist Safety][0][FS] gV al[Have Bike Lanes][0][PS] =⇒ gV al[Cyclist Safety][0][PS] gV al[Have Bike Lanes][1][FS] =⇒ gV al[Cyclist Safety][1][FS] gV al[Have Bike Lanes][1][PS] =⇒ gV al[Cyclist Safety][1][PS] ··· gV al[Have Bike Lanes][21][FS] =⇒ gV al[Cyclist Safety][21][FS] gV al[Have Bike Lanes][21][PS] =⇒ gV al[Cyclist Safety][21][PS] Appendix C. CSP Encoding 179

C.4 Evolving Functions

In the Evolving Functions procedure, we add constraints for the evolving functions by ﬁrst iterating over the evolving functions for each goal in EF . Then for each function, we iterate over the set of atomic functions and call the Atomic procedure to handle each instance of an atomic function.

1: procedure Evolving Functions(EF , tV al, gV al) 2: for all eF unction ∈ EF do 3: for all seg ∈ eF unction do 4: Atomic(tV al, seg)

Atomic Functions. After initialization to assign local variables from the input f, the Atomic procedure calls the appropriate helper procedure depending on the type of atomic function.

1: procedure Atomic(tV al, f) 2: g = f.goal . Get segment information. 3: type = f.type 4: x = f.x 5: start = f.start 6: stop = f.stop

7: t1 = tV al.get(start) . Add constraint for the order of start and stop.

8: t2 = tV al.get(stop) 9: CSP .add{

10: t1 < t2 11: } 12: if type = C then . Add constraints for function. 13: Constant(tV al, g, x, start, stop) 14: else if type = I then 15: Increase(tV al, g, x, start, stop) 16: else if type = D then 17: Decrease(tV al, g, x, start, stop) 18: else 19: Stochastic(tV al, g, x, start, stop)

The following four procedures describe the encoding of the atomic functions (i.e., Constant, Increase, Decrease and Stochastic). The Constant procedure assigns evaluation for g within the range [start, stop) to the reference evidence pair x (see Deﬁnition 3.2(a)).

1: procedure Constant(tV al, g, x, start, stop) 2: CSP .add{ 3: for all t ∈ tV al do 4: if (start ≤ t < stop) then 5: g[t] = x J K 6: } Appendix C. CSP Encoding 180

The Increase procedure constrains the evaluation for g within the range [start, stop) depending on the previous evaluation of g and the four possible values for the reference evidence pair x, which determine the maximum evidence pair for the intention g over the period [start, stop). Then for each pair of time points ti and tj in the period [start, stop), the Increase procedure constrains the evidence pair for g[tj] to be greater than or equal to the evidence pair for g[ti] (see Deﬁnition 3.2(b)).

1: procedure Increase(tV al, g, x, start, stop) 2: if x = (F, ⊥) then 3: CSP .add{

4: for all ti ∈ tV al do

5: for all tj ∈ tV al do

6: if (start ≤ ti < stop ∧ start ≤ tj < stop ∧ ti < tj ) then

7: (( g[ti] = (F, ⊥) =⇒ g[tj ] = (F, ⊥)) ∧ J K J K ( g[ti] = (P, ⊥) =⇒ g[tj ] = (F, ⊥) ∨ g[tj ] = (P, ⊥)) ∧ J K J K J K ( g[ti] = (⊥, P) =⇒ g[tj ] = (F, ⊥) ∨ g[tj ] = (P, ⊥) ∨ g[tj ] = (⊥, P)) ∧ J K J K J K J K ( g[ti] = (⊥, F) =⇒ g[tj ] = (F, ⊥) ∨ g[tj ] = (P, ⊥) ∨ g[tj ] = (⊥, P) ∨ g[tj ] = (⊥, F))) J K J K J K J K J K 8: } 9: else if x = (P, ⊥) then 10: CSP .add{

11: for all ti ∈ tV al do

12: for all tj ∈ tV al do

13: if (start ≤ ti < stop ∧ start ≤ tj < stop ∧ ti < tj ) then

14: (( g[ti] = (P, ⊥) =⇒ g[tj ] = (P, ⊥)) ∧ J K J K ( g[ti] = (⊥, P) =⇒ g[tj ] = (P, ⊥) ∨ g[tj ] = (⊥, P)) ∧ J K J K J K ( g[ti] = (⊥, F) =⇒ g[tj ] = (P, ⊥) ∨ g[tj ] = (P, ⊥) ∨ g[tj ] = (⊥, F))) J K J K J K J K 15: } 16: else if x = (⊥, P) then 17: CSP .add{

18: for all ti ∈ tV al do

19: for all tj ∈ tV al do

20: if (start ≤ ti < stop ∧ start ≤ tj < stop ∧ ti < tj ) then

21: (( g[ti] = (⊥, P) =⇒ g[tj ] = (⊥, P)) ∧ J K J K ( g[ti] = (⊥, F) =⇒ g[tj ] = (⊥, P) ∨ g[tj ] = (⊥, F))) J K J K J K 22: } 23: else if x = (⊥, F) then 24: CSP .add{

25: for all ti ∈ tV al do

26: for all tj ∈ tV al do

27: if (start ≤ ti < stop ∧ start ≤ tj < stop ∧ ti < tj ) then

28: ( g[ti] = (⊥, F) =⇒ g[tj ] = (⊥, F)) J K J K 29: } Appendix C. CSP Encoding 181

The Decrease procedure constrains the evaluation for g within the range [start, stop) depending on the previous evaluation of g and the four possible values for the reference evidence pair x, which determine the minimum evidence pair for the intention g over the period [start, stop). Then for each pair of time points ti and tj in the period [start, stop), the Decrease procedure constrains the evidence pair for g[tj] to be less than or equal to the evidence pair for g[ti] (see Deﬁnition 3.2(c)).

1: procedure Decrease(tV al, g, x, start, stop) 2: if x = (⊥, F) then 3: CSP .add{

4: for all ti ∈ tV al do

5: for all tj ∈ tV al do

6: if (start ≤ ti < stop ∧ start ≤ tj < stop ∧ ti < tj ) then

7: (( g[ti] = (⊥, F) =⇒ g[tj ] = (⊥, F)) ∧ J K J K ( g[ti] = (⊥, P) =⇒ g[tj ] = (⊥, F) ∨ g[tj ] = (⊥, P)) ∧ J K J K J K ( g[ti] = (P, ⊥) =⇒ g[tj ] = (⊥, F) ∨ g[tj ] = (⊥, P) ∨ g[tj ] = (P, ⊥)) ∧ J K J K J K J K ( g[ti] = (F, ⊥) =⇒ g[tj ] = (⊥, F) ∨ g[tj ] = (⊥, P) ∨ g[tj ] = (P, ⊥) ∨ g[tj ] = (F, ⊥))) J K J K J K J K J K 8: } 9: else if x = (⊥, P) then 10: CSP .add{

11: for all ti ∈ tV al do

12: for all tj ∈ tV al do

13: if (start ≤ ti < stop ∧ start ≤ tj < stop ∧ ti < tj ) then

14: (( g[ti] = (⊥, P) =⇒ g[tj ] = (⊥, P)) ∧ J K J K ( g[ti] = (P, ⊥) =⇒ g[tj ] = (⊥, P) ∨ g[tj ] = (P, ⊥)) ∧ J K J K J K ( g[ti] = (F, ⊥) =⇒ g[tj ] = (⊥, P) ∨ g[tj ] = (P, ⊥) ∨ g[tj ] = (F, ⊥))) J K J K J K J K 15: } 16: else if x = (P, ⊥) then 17: CSP .add{

18: for all ti ∈ tV al do

19: for all tj ∈ tV al do

20: if (start ≤ ti < stop ∧ start ≤ tj < stop ∧ ti < tj ) then

21: (( g[ti] = (P, ⊥) =⇒ g[tj ] = (P, ⊥)) ∧ J K J K ( g[ti] = (F, ⊥) =⇒ g[tj ] = (P, ⊥) ∨ g[tj ] = (F, ⊥))) J K J K J K 22: } 23: else if x = (F, ⊥) then 24: CSP .add{

25: for all ti ∈ tV al do

26: for all tj ∈ tV al do

27: if (start ≤ ti < stop ∧ start ≤ tj < stop ∧ ti < tj ) then

28: ( g[ti] = (F, ⊥) =⇒ g[tj ] = (F, ⊥)) J K J K 29: } Appendix C. CSP Encoding 182

The Stochastic procedure ensures that each evaluation for g within the range [start, stop) is not assigned an evidence pair from the Weak Conﬂict set (see Deﬁnition 3.2(d)).

1: procedure Stochastic(tV al, g, x, start, stop) 2: CSP .add{ 3: for all t ∈ tV al do 4: if (start ≤ t < stop) then 5: ( g[t] = (F, ⊥) ∨ g[t] = (P, ⊥)∨ J K J K g[t] = (⊥, P) ∨ g[t] = (⊥, F)) J K J K 6: }

Calling Evolving Functions creates constraints for each evolving function. We illustrate the

hHave Design, hDenied-Satisfied,(F, ⊥), {t0, tdesign, tend}ii function below. Note the Denied- Satisfied function is composed of two atomic functions:

{hConstant,(⊥, F), t0, tdesigni, hConstant,(F, ⊥), tdesign, tendi}

(t0 ≤ tV al[0] ∧ tV al[0] < tdesign) =⇒ gV al[Have Design][0][FD]

(t0 ≤ tV al[1] ∧ tV al[1] < tdesign) =⇒ gV al[Have Design][1][FD] ···

(t0 ≤ tV al[21] ∧ tV al[21] < tdesign) =⇒ gV al[Have Design][21][FD]

(tdesign ≤ tV al[0] ∧ tV al[0] < tend) =⇒ gV al[Have Design][0][FS]

(tdesign ≤ tV al[1] ∧ tV al[1] < tend) =⇒ gV al[Have Design][1][FS] ···

(tdesign ≤ tV al[21] ∧ tV al[21] < tend) =⇒ gV al[Have Design][21][FS] Appendix C. CSP Encoding 183

C.5 NotBoth Evolving Function

We encode the NotBoth evolving function (see Deﬁnition 3.9) with the Not Both Constraint procedure, which takes as input tV al, a gV al for each goal involved, a reference time point tRef and the type of function (i.e., NBN or NBD). The procedure adds constraints to each time point.

1: procedure Not Both Constraint(tV al, gV al, r)

2: g1 = r.g1

3: g2 = r.g2 4: type = r.subT ype

5: tRef = r.timeP oint 6: if type = NBN then 7: fV al = (⊥, ⊥) 8: else 9: fV al = (⊥, F) 10: for all t ∈ tV al do 11: CSP .add{

12: if t < tRef then . Prior to tRef .

13: gV al[g1][t] = (⊥, ⊥) ∧ gV al[g2][t] = (⊥, ⊥)

14: if t = tRef then . At tRef .

15: (gV al[g1][t] = (F, ⊥) ∧ gV al[g2][t] = fV al)∨

(gV al[g1][t] = fV al ∧ gV al[g2][t] = (F, ⊥))

16: if t > tRef then . After tRef .

17: gV al[g1][t] = gV al[g1][tRef ]∧

gV al[g2][t] = gV al[g2][tRef ] 18: }

Calling Not Both Constraint creates constraints for each intention at each time point. We NBN illustrate the Permanent Construction ←−−−→ Temporary Construction relationship (see Figure 3.5). . tdecision

(tV al[0] < tRef ) =⇒ gV al[Permanent Construction][0] = (⊥, ⊥)

(tV al[1] < tRef ) =⇒ gV al[Permanent Construction][1] = (⊥, ⊥) ···

(tV al[21] < tRef ) =⇒ gV al[Permanent Construction][21] = (⊥, ⊥)

(tV al[0] < tRef ) =⇒ gV al[Temporary Construction][0] = (⊥, ⊥)

(tV al[1] < tRef ) =⇒ gV al[Temporary Construction][1] = (⊥, ⊥) ···

(tV al[21] < tRef ) =⇒ gV al[Temporary Construction][21] = (⊥, ⊥) Appendix C. CSP Encoding 184

(gV al[Permanent Construction][tRef ] = (F, ⊥) ∧ gV al[Temporary Construction][tRef ] = (⊥, ⊥))∨

(gV al[Permanent Construction][tRef ] = fV al ∧ gV al[Temporary Construction][tRef ] = (F, ⊥))

(tV al[0] > tRef ) =⇒ gV al[Permanent Construction][0] = gV al[Permanent Construction][tRef ]

(tV al[1] > tRef ) =⇒ gV al[Permanent Construction][1] = gV al[Permanent Construction][tRef ] ···

(tV al[21] > tRef ) =⇒ gV al[Permanent Construction][21] = gV al[Permanent Construction][tRef ]

(tV al[0] > tRef ) =⇒ gV al[Temporary Construction][0] = gV al[Temporary Construction][tRef ]

(tV al[1] > tRef ) =⇒ gV al[Temporary Construction][1] = gV al[Temporary Construction][tRef ] ···

(tV al[21] > tRef ) =⇒ gV al[Temporary Construction][21] = gV al[Temporary Construction][tRef ]

C.6 Constraints between Time Points

We encode constraints between time points directly into CSP by iterating over each constraint adding it’s equivalent in CSP. 1: procedure Time Point Constraints(MC, tV al) 2: for all c ∈ MC do

3: t1 = tV al.get(c.t1)

4: t2 = tV al.get(c.t2) 5: type = c.T ype 6: val = c.V alue

7: if t2 = null then 8: CSP .add{

9: t1 = c.V alue 10: } 11: else if type = ‘ <0 then 12: CSP .add{

13: t1 < t2 14: } 15: else 16: CSP .add{

17: t1 = t2 18: }

Calling Time Point Constraints creates constraints for each of the elements in MC. Here we

illustrate the constraint added for tdesign < tdecision (see the speciﬁcation in Figure 3.5):

tV al[tdesign] < tV al[tdecision]

C.7 Conﬂict Avoidance

The levels of conflict avoidance (see Definition 2.9) specify which values should be avoided in the evaluation of goals. Prevent Conflicts adds the constraints required for conflict avoidance, according to Appendix C. CSP Encoding 185 the CFLevel given, by constraining the evidence predicates for each goal.

1: procedure Prevent Conflicts(CFLevel, gV al) 2: for all g ∈ gV al do 3: for all t ∈ gV al[g] do 4: if CFLevel = Strong Conflict then 5: CSP .add{ 6: ¬gV al[g][t][FD] ∨ ¬gV al[g][t][FS] 7: } 8: else if CFLevel = Medium Conflict then 9: CSP .add{ 10: (¬gV al[g][t][FS] ∨ ¬gV al[g][t][PD])∧ 11: (¬gV al[g][t][PS] ∨ ¬gV al[g][t][FD]) 12: } 13: else if CFLevel = Weak Conflict then 14: CSP .add{ 15: ¬gV al[g][t][PD] ∨ ¬gV al[g][t][PS] 16: }

Calling Prevent Conflicts creates constraints for each intention in the model depending on the conflict avoidance level (see Definition 2.9) . If Strong Conflict was selected, the Bike Lane Usage would have the following additional constraints.

¬(gV al[Bike Lane Usage][0][FD]) ∨ ¬(gV al[Bike Lane Usage][0][FS]) ¬(gV al[Bike Lane Usage][1][FD]) ∨ ¬(gV al[Bike Lane Usage][1][FS]) ··· ¬(gV al[Bike Lane Usage][21][FD]) ∨ ¬(gV al[Bike Lane Usage][21][FS])

C.8 User Evaluations

Finally, we add the constraints for the User Evaluations. The User Affirm procedure considers each element in UEval and adds a single constraint for the goal value in CSP.

1: procedure User Affirm(UEval, tV al, gV al) 2: for all u ∈ UEval do 3: g = u.goal 4: t = u.time 5: e = u.eval 6: CSP .add{ 7: gV al[g][t] = e 8: } Appendix C. CSP Encoding 186

In the Bike Lanes example, the User Affirm procedure encodes the user evaluations

{hHave Design, ta2 ,(F, ⊥)i, hCyclist Safety, t0,(⊥, P)i} as the following:

gV al[Have Design][ta2 ] = (F, ⊥)

gV al[Cyclist Safety][t0] = (⊥, P)

C.9 Encoding and Solving an Evolving Goal Graph

Recall from Section C.1 that our goal was to encode the problem of finding a Graph Evaluation Path into a constraint satisfaction problem (CSP). Now that we have constructed the pieces, we put them together. Algorithm C.1 takes as input a goal graph and a set of graph time points and a set of user evaluations for the given graph, as well as an optional conflict level. The algorithm then encodes these inputs as a CSP and finds satisfying assignments (if feasible) to return a Time Point Path (see Definition 4.4) and a Graph Evaluation Path (see Definition 4.5). Line 1 of Algorithm C.1 creates a new store for the CSP solver. Line 2–3 creates the data structure for the time point path and adds constraints between time points in the path (additional constraints are added in the Evolving Functions procedure on line 8). Lines 4-8 create the data structure for the goals in the graph and add all the constrains over the goals. Note that the CSP object is global to all procedures so we don’t need to pass it to every procedure that creates constraints. Lines 9–11 call CSP to solve the graph, meaning the CSP solver finds evaluations for each time point and all of the goals at each time point. If a solution is found, the Decode function matches the evaluations with the goals and time points. If no solution is found, Algorithm C.1 returns null. Appendix C. CSP Encoding 187

Algorithm C.1 Encode as CSP & Compute Time Point Path and Goal Graph Evaluation Input: Goal Graph hA, G, R, EF , MC, maxTimei . Satisfying Definition 3.11. A Set of Graph Time Points TP (M) . Definition 4.1 A set of User Evaluations UEval . Definition 4.2 (Optional) A Conflict Level CFLevel . Definition 2.9 Output: Time Point Path (satisfying Definition 4.4) and Goal Graph Evaluation Path (satisfying Defini- tion 4.5) or null indicating infeasibility.

1: CSP = new CSP Solver Store . Global object to all procedures. 2: tV al = Initialize Time Points(TP (M), maxT ime) . First Encoding Procedure 3: Time Point Constraints(MC, tV al) 4: gV al = Initialize Goals(G, tV al) 5: Prevent Conflicts(CFLevel, gV al) 6: User Affirm(UEval, tV al, gV al) 7: Link Constraints(G, R, tV al, gV al) 8: Evolving Functions(EF , tV al, gV al) . Last Encoding Procedure 9: if ¬CSP .SOLV E(tV al, gV al) then . Find satisfying assignments to given variables. 10: return null . No Solution 11: return Decode(TP (M), G, tV al, gV al) . Get a return solution.

12: function Decode(TP (M), G, tV al, gV al) 13: tResult = {} 14: for all t ∈ TP (M) do 15: tResult.add(ht, tV al[t.index]i) 16: gResult = {} 17: for all t ∈ TP (M) do 18: for all g ∈ G do 19: gResult.add(ht, g, gV al[g.index][t.index]i) 20: return htResult, gResulti Index

Atomic Functions, 36, 37, 40, 54

Backward Analysis, 4, 6, 7, 10, 16–18, 60 Backward Propagation, see Backward Analysis Bike Lanes example (BLE), 6, 9, 19–21, 23, 26, 27, 32, 33, 35, 37–39, 42, 43, 47, 49–56, 58, 66–71, 78, 79, 129, 130, 134, 137, 171, 186 BloomingLeaf, see Tools

Constraint Satisfaction Problem, see CSP CSP, 75, 86, 90, 91, 107, 170, 171, 176, 184–187

Early-phase Requirements Engineering, ii, 3, 8, 9, 12, 107 Evolving Intentions (EIs), ii, 9–11, 35, 49, 70, 71, 73, 87, 88, 93–100, 102, 103, 106, 107, 109, 129–131 Evolving Intentions Framework, 9–11, 35, 44–47, 49, 60, 66, 70, 73, 74, 78, 85–87, 107, 129–132 Evolving Relationships, 40, 45

Forward Analysis, 4, 5, 7, 10, 16, 17, 60, 93, 96, 98, 99, 102 Forward Propagation, see Forward Analysis

Goal-oriented Requirements Engineering (GORE), 4, 12, 26, 86, 124, 134–140 Graph Constraints, 42, 51, 53, 86 Graph Evaluation Path, 49, 53–57, 59–62, 130, 170, 171, 186, 187 GrowingLeaf, see Tools iStar (i*), ii, 4, 5, 8–16, 18–20, 33, 35, 45–47, 49, 50, 60, 63, 70, 73, 74, 78, 86–90, 94, 96, 97, 106, 107, 129–131

JaCoP, 77, 86, 170 Java, 77, 82, 83, 88

Leaf-level Intentions, 4, 10, 16, 18, 28, 49, 50, 60–63, 67, 70, 89, 130

NotBoth Function, 41–46, 54–56, 67, 86, 170, 173, 174, 183

Related Work, 7, 70, 86, 107, 126 Repeated Forward Analysis (Rep-FA), 93, 94, 96, 97, 102–104, 107 Research Questions, 9, 93, 94, 110

188 INDEX 189

Root-level Intentions, 4–7, 10, 16, 18, 51, 63–66, 69, 89, 90

Simulation over All Evolving Intentions (EI-Sol), 10, 53, 60, 75, 81, 86, 87 Simulation over All Evolving Intentions with Path Histories, 86 Simulation over Evolving Leaf Intentions (EI-Sim), 9–11, 61, 74, 86, 87, 93–97, 102–104, 106–108 Simulation over Stochastically Evolving Intentions (SEI-Sim), 93–97, 102–104, 106, 107 Simulation Strategies, ii, 10, 49, 50, 61, 70, 73, 87, 130 ASR1, 62, 63, 66, 73, 74, 87, 89–92, 107 ASR2, 62–64, 66, 67, 69, 71, 73, 75, 81, 90–92, 116, 170 ASR3, 62, 63, 66, 73–75, 87, 90–92, 107 Spadina Expressway, 11, 109–119, 122–124, 126, 130, 131 Full Model, 112, 113, 118, 119, 122, 123, 126 Scene I, 112–116, 118–121 Scene II, 112, 113, 115, 116, 118–121, 124, 125 Scene III, 112, 113, 116–119, 125 Stochastically Evolving Intentions (SEIs), 93–99, 102, 107

Time Point Path, 49, 53–57, 59–62, 64, 171, 186, 187 Tools BloomingLeaf, ii, 10, 11, 53, 73, 78–82, 84–86, 112, 125, 130, 132 GrowingLeaf, ii, 10, 11, 46, 64, 73–78, 85–90, 93–95, 97, 99, 104–108, 130, 153 GrowingLeaf-EI-Sim (Tool-EI), 95–97 GrowingLeaf-Forward Analysis (Tool-FA), 96, 97 GrowingLeaf-SEI-Sim (Tool-SEI), 95–97 Toronto Spadina Expressway Project, see Spadina Expressway Tropos, ii, 4, 6, 7, 9–12, 14, 19, 20, 33, 35, 36, 45, 47, 49, 50, 60, 70, 73, 78, 109, 112, 116, 126, 129–131

User Evaluations, 26, 29–31, 50–55, 57–62, 64, 69, 86, 170, 185–187 User-Deﬁned Function (UD), 39, 40, 42, 43, 45, 46, 51, 52, 56, 58, 86, 88, 89, 100, 114, 120, 121, 170

Videos, 95–97, 116, 118, 119, 122

Waste Management example (WME), 5, 9, 12, 13, 15, 33, 35, 45, 47–49, 60, 61, 63, 66, 67, 70, 74, 90, 95, 129, 130, 153