Problem Structuring Methods in System Dynamics Modeling: a Cognitive Fit Perspective

Arash Golnam, Gorica Tapandjieva, Sérgio Viana, Alain Wegmann

1Ecole Polytechnique Fédérale de Lausanne School of Computer and Communication Sciences (I&C) Systemic Modeling Laboratory (LAMS) Station 14, CH-1015 Lausanne, Switzerland Tel: +41 21 693 43 81 Email {arash.golnam, sergio.viana, alain.wegmann}@epfl.ch

Abstract

The knowledge residing in the mental models of clients and stakeholders is considered as a crucial source of information by system dynamicists. Despite the importance of this knowledge, the theory of SD does not provide practical means for eliciting and recording it. Thus, several methods known as problem structuring methods (PSMs) have been developed and employed in the SD community to facilitate the problem situation conceptualization in group model building (GMB). Despite the growing body of literature on the application of the PSMs, limited work has been done on assessing and comparing the relative usefulness of PSMs in terms of their potential impact on problem-solving performance. In this paper, by invoking Cognitive Fit Theory, we develop a set of propositions for the analysis of the impact of PSMs on problem-solving performance.

Keywords: Cognitive Fit Theory, Problem-solving Performance, Problem Structuring Methods (PSMs), System Dynamics (SD)

1 1. Introduction

In system dynamics (SD) modeling, the information contained in the stakeholders’ and clients’ mental models is considered as a crucial source of information Forrester (1961, 1992). In effect, system dynamics is emerging as a methodology in which simulation models are built in close cooperation with clients, in order to elicit and capture the knowledge in the mental models of the client group, required for the model building effort to improve decision making or problem- solving performance. In the SD literature “group model building” (GMB) (Andersen et al. 1997; Vennix 1996) is used to capture clients’ and stakeholders’ participation in constructing SD models that can serve to develop policies that improve problem situations.

By involving the client and stakeholder groups and facilitating collaborative modeling efforts with these parties, GMB is expected to create a more shared perspective on the problem and on potential solutions aimed at improving the situation (Richardson and Andersen 1995; Vennix et al. 1996; Vennix 1996) through surfacing implicit assumptions and beliefs held by the clients and stakeholders.

Richardson and Pugh (1981) outline the SD modeling phases as a seven-step process that begins with identification of the problem and system conceptualization. In the conceptualization phase of systems modeling in general or SD model building in particular, the modeler observes some aspect of reality referred to as the “universe of discourse” (UoD). The modeler then tries to distinguish a set of entities that compose the universe of discourse and the relationships between them. Conceptualizations are in effect, a lens through which the modeler observes phenomena of interest in a UoD (Tarski and Corcoran 1983). Next, the modeler develops a model in the representation domain. The model is composed of modeling constructs that represent the observed entities in the UoD. The conceptualization explains the kinds of modeling constructs in the representation domain and allows a mapping between the modeling constructs in the representation domain and the entities observed in the universe of discourse. A conceptualization thereby gives the modeling constructs a real-world interpretation. Figure 1 represents the process of systems modeling. In essence, the conceptualization phase is a phase through which the SD model is generated and evolves.

With respect to SD modeling, as stated, in the retrospect (prior to the emergence of GMB), it was the SD expert who formed a conceptualization of reality whereas, currently the clients or the stakeholders who are, in effect, the direct participants in the problem situation, are involved in the mapping of and translating the phenomenon of interest or the problem situation existing in “reality” to the SD “modeling constructs” in the representation domain. This involvement results in constructing a model that gives a better representation of reality (Lane and Oliva 1998).

2

Figure 1: Conceptualizations in the Modeling Process

Despite the fact that the power of SD originates from using the information obtained from the clients and stakeholders, the SD literature offers limited insights and implications into the ways of eliciting and recording this information in order to develop conceptualizations of the problem situation in the SD modeling process (Lane and Oliva 1998; Lane 1994). In other words, the theory of SD does not address the practical means of helping clients and stakeholders jointly surface and articulate their views of the problem situation. A better conceptualization of the phenomenon of interest in reality will, no doubt, inspire better modeling constructs upon which a SD model may center.

This theoretical gap in problem-solving with SD has led to an increasing interest in the SD community to develop or employ tools that can aid and facilitate the conceptualization phase of the model building. A wide diversity of tools and techniques have been developed inside the SD community that can assist in eliciting the knowledge residing in the clients and stakeholders’ mental models in order to aid model conceptualizations. Such tools and techniques are referred to as problem structuring methods (PSMs) (Rosenhead 1996; Mingers and Rosenhead 2004) in the Soft OR and SD literature. Among PSMs, Cognitive Maps (Eden 1994) and (magnetic) Hexagons (Lane 1993; Hodgson 1992) are developed in the SD community.

A number of hybrid approaches, integrating elements of system dynamics and particular PSMs, have also emerged to support the problem conceptualization. Lane and Oliva (1998), for example, develop a theoretical framework (i.e. Holon Dynamics) that integrates system dynamics and soft systems methodology. Lane (1994) gives an elaborate account of PSMs.

Knowing that a wide variety of techniques and approaches exist, now the key research question is how to compare and assess the methods in terms of their potential contribution to the performance of the SD modeling process. Despite the growing interest towards the application of methods and techniques that facilitate the conceptualization phase of SD modeling, limited work

3 has been found that proposes (theoretical) insights to discuss and analyze this important aspect of employing such methods.

Applying a proper technique for problem structuring or to aid the joint conceptualization of the problem situation can give rise to active involvement of clients and stakeholders in the model building process and improve of the quality of the communication within the clients and between the clients and the SD modeling experts and thereby creates opportunities for learning and improving the mental models of the clients and stakeholders that can serve to clarify the problem situation (Akkermans and Vennix 1997).

SD modeling process involves a wide variety of cognitive tasks that are carried out by the SD modeling experts and/or clients (Eden 1994; Lane 1994; Lane and Oliva 1998; Rouwette, et al. 2008). Hence, adopting a cognitive view, in this paper we view SD model building as a dual task problem-solving consisting of “conceptualization of the problem situation”, generally facilitated by the SD modeling expert and carried out by clients and stakeholders, and “constructing the SD model” carried out by the SD modeling expert. This perspective of SD modeling enables us to invoke the Cognitive Fit Theory in dual task problem-solving (Vessey 1991; Shaft and Vessey 2006). Cognitive Fit Theory provides useful insights into how matching internal and external representations in dual task problem-solving to the problem-solving task can lead to a higher performance of problem-solving employing SD model building. In other words, Cognitive Fit Theory explains how the gap between the generalist thinking of the clients and stakeholders and the specialism of SD modelers can be bridged to increase the problem-solving performance.

Figure 2: Research Design

In Figure 2, we show our research design to address our research question. As illustrated, we apply the Cognitive Fit Theory to the system dynamics modeling process to derive propositions on how the PSMs can lead to problem-solving performance in the SD modeling process. These propositions are then translated to a set of hypotheses represented as the characteristics of an ideal PSM that can increase the problem solving performance with SD. As this work is a research in progress, the hypotheses have not been tested yet. We intend to re-define the hypotheses from the perspective of visual notation analysis and validate them through user study (Moody 2009).

4 In order to present our findings and argument, this paper is structured in the following way. In section 2 we briefly discuss the underlying theory of Cognitive Fit in dual task problem-solving, delineating the role of external and internal representation in problem-solving. Then, we develop a view of SD modeling based on Cognitive Fit Theory. Building on the theoretical insights developed in Section 2, in Section 3, we present a number of propositions on how the PSMs can lead to problem-solving performance in the SD modeling process. These propositions provide us with the tentative characteristics of a PSM that can improve problem-solving performance. Section 4 includes our future work and conclusions.

2. SD Modeling from the Perspective of Cognitive Fit Theory

Cognitive fit (Vessey 1991) is a theory that can be employed to shed light on the cognitive processes applied to carryout tasks such as constructing SD models. It proposes that the correspondence between the problem-solving task and the format of the representation of the problem leads to superior task performance for individual users. Vessey (1991) suggest that for carrying out a problem-solving task, individuals create mental representation of the problem using the characteristics of both the representation of the problem and the problem-solving task (Vessey 1991). Vessey (1991) defines mental representation as “the way the problem is represented in human working memory”. When individuals need to solve problems in a certain domain their performance will be enhanced if the representation of the problem matches the problem-solving task. Figure 3, illustrates the Cognitive Fit Theory.

Figure 3: Cognitive Fit Model for Problem-Solving (Vessey, 1991)

From the perspective of cognitive fit, SD model building entails two distinct tasks; one related to the problem situation conceptualization and the other related to SD model construction. Thus, we are faced with a dual task problem-solving. Looking at the SD modeling as a dual task problem- solving enables us to understand the inter-relationships between the tasks and the cognitive processes acting on them.

Shaft and Vessey (2006) extended the Cognitive Fit Theory for dual task problem-solving. They explain: A dual task occurs when problem solvers perform two (or more) tasks simultaneously where each task is referred to as subtask.

5 Shaft and Vessey modified their model based on the work of Zhang and Norman (1994) and Zhang (1997) on distributed cognition and included the concepts of external and internal representations. See Figure 4.

Figure 4: Extended Cognitive Fit Model for Dual Task Problem-Solving (Vessey,1996) Zhang (1997) defines internal and external representations in the following way:

Internal representations are the knowledge and structure in memory, as propositions, productions, schemas, neural networks, or other forms. The information in internal representations has to be retrieved from memory by cognitive processes, although the cues in external representations can sometimes trigger the retrieval processes.

External representations are “the knowledge and structure in the environment, as physical symbols, objects, or dimensions (e.g., written symbols, beads of abacuses, dimensions of a graph, etc.), and as external rules, constraints, or relations embedded in physical configurations (e.g., spatial relations of written digits, visual and spatial layouts of diagrams, physical constraints in abacuses, etc.)”. For example, problem solvers use external representations when they use a list for grocery shopping or when they use graphs to understand economic trends.

Dual task problem-solving model reflects the fact that both the internal and external representations, and the interactions among them, contribute to the mental representation for task solution developed to solve the problem. Zhang explains the interaction between the internal and external representation by considering a multiplication as a problem-solving task. Zhang (1997) explains: “Let us consider multiplying 735 by 278 using paper and pencil. The internal representations are the meanings of individual symbols (e.g., the numerical value of the arbitrary symbol "7" is seven), the addition and multiplication tables, arithmetic procedures, etc., which have to be retrieved from memory; the external representations are the shapes and positions of the symbols, the spatial relations of partial products, etc., which can be perceptually inspected from the environment (Zhang and Norman, 1994). To perform this task, people need to process the information perceived from external representations and the information retrieved from internal representations in an interwoven, integrative, and dynamic manner.”

Figure 5 represents SD model building as a dual task problem-solving from the perspective of cognitive theory. As illustrated, the performance of the SD models in terms of capturing and serving to develop policies that improve the problem situation depends on carrying out “problem

6 situation conceptualization” and “SD model construction” subtasks. Meaning, the stakeholders and clients apply a PSM to develop a conceptualization of the problem situation. This PSM serves as an external problem representation in the problem situation conceptualization sub-task. Then, thanks to this external representation, the stakeholders form internal representations of the problem situation. These two representations iterate until a shared mental representation of the problem situation is achieved among the stakeholders and the problem situation is conceptualized. This conceptualization is then applied by the SD modeler in the SD model construction subtask to build the SD model.

Figure 5: Cognitive View of SD Modeling as a Dual Task Problem-solving

3. Implications for Assessing the Impact of PSMs on Improving the SD Modeling Performance

In this section, based on the insights gained from the cognitive fit view of SD modeling, we formulate a number of propositions on how the tools and techniques applied to conceptualize the problem situation can in fact improve the problem-solving performance of the SD models.

7

Vessey (1991) states that:

“Matching representation to tasks leads to the use of similar problem-solving processes, and hence the formulation of a consistent mental representation. There will be no need to transform the mental representation . . . to extract information from the problem representation and to solve the problem. Hence, problem-solving with cognitive fit leads to effective and efficient problem-solving performance."

To, when the type of information emphasized in the internal and the external representations, does not match the knowledge emphasized in the problem-solving tasks (i.e. Problem Situation Conceptualization and SD Model Construction), there is nothing to guide the SD Modeler or the Clients/Stakeholders in working toward task solution, and (s)he must exert greater cognitive effort to transform the information into a form suitable for solving that particular type of problem specified in the task (Vessey and Galletta 1991). This excessive effort will result in decreased problem-solving performance.

The above statements can be translated into the following propositions:

Proposition 1a: SD models results in a higher performance in problem-solving if, the format and the type of the knowledge emphasized in the PSM as an external representation in the Problem Situation Conceptualization Subtask triggers the elicitation and the retrieval processes of the tacit knowledge, mental models and causal structures residing in the stakeholders’ mind.

Proposition 1b: SD models results in a higher performance in the problem-solving if, the format and the type of the knowledge captured in the problem situation conceptualization, employed by the SD modeler as an external representation, matches the format of knowledge emphasized in “SD model construction” subtask.

The presence of the abovementioned conditions avoids: • Transforming the mental representation (i.e. the interaction between the internal and the external representations) of the problem situation to match (the format of) “problem situation conceptualization” subtask. • Transforming the “problem situation conceptualization” subtask to match (the format of) “SD model construction” subtask.

Moreover, researchers in the field of distributed cognition have long highlighted the importance of graphical representations such as diagrams and pictures as special forms of external representations that can contribute to the problem-solving performance.

8

Zhang (1997), Scaife and Rogers (1996) emphasize the importance of external representations, in particular graphical representations, by asserting that external representations such as diagrams and pictures can give individuals access to knowledge and skills that can not be accessed by internal representations. In this respect Larkin and Simon (1987), in their seminal paper entitled “Why a Diagram is (Sometimes) Worth Ten Thousand Words” present their arguments on why a diagram can be superior to other forms of external representation such as sentential or verbal representations.

Larkin & Simon (1987) assert the following reasons to support their argument: • Diagrams facilitate the problem-solving by assembling all pieces of information and thereby reducing the time required to make inferences • Diagrams inherently support a substantial number of perceptual inferences, which are uncomplicated for humans. • Diagrams support cognitive operators that can recognize features easily and make inferences directly.

Considering the problem situation conceptualization as a form of external representation used by the SD modelers in the “SD model construction” subtask, we can thereby formulate the following proposition:

Proposition 2: The more graphical the conceptualization of the problem situation the higher the performance of problem-solving.

The propositions formulated in this section provide us with some insights into assessing and analyzing the potential contribution of a PSM to the problem-solving performance. Based on the formulated oppositions we can come to a tentative conclusion that a PSM can contribute to the performance of problem-solving by SD methodology, when it fulfills the following conditions:

1) It is graphical (diagrammatic) rather than sentential, to result in a better cognitive fit. 2) It is lightweight, to be well and commonly understood by stakeholders. 3) It is a multi-viewpoint method, to capture the wide range of perspectives held by various stakeholders. 4) It has modeling constructs that are close to constructs in the stock and flow diagram (or perhaps CLDs), to avoid the re-representation and re-interpretation of the conceptualization by the SD modeler.

4. Conclusion and Future Work

In this paper, we applied a theoretical perspective from cognitive science to gain an understanding of how PSMs can improve problem-solving with system dynamics modeling. We developed a number of propositions to explain the inter-relation between the PSM and problem-

9 solving performance with SD in GMB. These propositions guided us in enumerating a number of characteristics for a PSM that can increase problem-solving performance.

Our current research involves applying principles from visual notation analysis to gain a better understanding of the PSMs that are applied in model building with SD. In order to test the hypotheses stated in the previous section, we analyze the PSMs applied in the SD community such as magnetic hexagons, cognitive maps etc. from the perspective of their visual notation anatomy (Moody 2009). The anatomy of a modeling method is composed of three elements; the constructs (i.e. the vocabulary), the semantics and the grammar. This understanding will assist us in conducting a user study in order to test the validity of the hypotheses developed.

In a parallel stream of research, we are in the process of designing a PSM for the group model building with SD. Our PSM is based on the Systemic Enterprise Architecture Methodology (SEAM) (Wegmann et al. 2007). SEAM is a modeling technique that has so far been developed at École Polytechnique Fédérale de Lausanne (EPFL), SEAM has been used for teaching (Wegmann et al. 2007) and consulting (Wegmann et al. 2005) mainly in the field of enterprise architecture and requirements engineering since 2001.

The foundations of SEAM are in General Systems Thinking (GST) (Weinberg 2001), organizational Cybernetics (Beer 1979,1984) and in RM-ODP (ISO Standard 1995). GST is the study of principles that are applicable to any kind of system (e.g. business system, etc.) and Cybernetics is defined as the science of effective organization (Beer 1979). RM-ODP is a software engineering ISO standard that provides solid definitions for the SEAM concepts (e.g. process, state, property). SEAM is rigorously defined based on these systemic and software engineering concepts. SEAM federates multiple modeling techniques (such as discrete behavior, goals or quantitative models). (Golnam et. al, 2010) is our first effort to explore the applicability of SEAM as a PSM in SD modeling process. The results of our first stream of research inform us about the improvements and the modifications that should be made to SEAM, so that it results in a cognitive fit when applied in the SD modeling process.

References

Akkermans, H. A, and J. A.M Vennix. 1997. Clients' opinions on group model-building: an exploratory study. System Dynamics Review 13, no. 1: 3–31. Andersen, D. F, G. P Richardson, and J. A.M Vennix. 1997. Group model building: adding more science to the craft. System Dynamics Review 13, no. 2: 187–201. Beer, S. 1979. The heart of enterprise. John Wiley & Sons. ———. 1984. The viable system model: its provenance, development, methodology and pathology. Journal of the Operational Research Society 35, no. 1: 7–25. Eden, C. 1994. Cognitive mapping and problem structuring for system dynamics model building. System Dynamics Review 10, no. 2: 257–276. Forrester, J. W. 1961. Industrial dynamics. Pegasus Communications, Waltham, MA. Golnam, Arash, Ann Van Ackere, and Alain Wegmann. 2010. Integrating System Dynamics

10 and Enterprise Modeling to Address Dynamic and Structural Complexities of Choice Situations. In . Seoul, Korea: System Dynamics Society, July 25. Hodgson, A. M. 1992. Hexagons for systems thinking. European Journal of Operational Research 59, no. 1: 220–230. ISO Standard. 1995. ISO/IEC 10746-1, 2, 3, 4| ITU-T Recommendation X. 901, X. 902, X. 903, X. 904.“. ITU. Google Scholar. Lane, D. C. 1993. The road not taken: observing a process of issue selection and model conceptualization. System dynamics review 9, no. 3: 239–264. ———. 1994. With a little help from our friends: how system dynamics and soft OR can learn from each other. System Dynamics Review 10, no. 2: 101–134. Lane, D. C, and R. Oliva. 1998. The greater whole: Towards a synthesis of system dynamics and soft systems methodology. European Journal of Operational Research 107, no. 1: 214–235. Larkin, J. H, and H. A Simon. 1987. Why a Diagram is (Sometimes) Worth Ten Thousand Words. Cognitive science 11, no. 1: 65–100. Le, L., and A. Wegmann. 2006. SeamCAD: Object-Oriented Modeling Tool for Hierarchical Systems in Enterprise Architecture. In Hawaii International Conference on System Sciences, 39:179. Mingers, J., and J. Rosenhead. 2004. Problem structuring methods in action. European Journal of Operational Research 152, no. 3: 530–554. Moody, L.D. 2009. The “Physics” of Notations: Toward a Scientific Basis for Constructing Visual Notations in Software Engineering. IEEE Transactions on Software Engineering 35, no. 6: 756-779. Richardson, G. P, and D. F Andersen. 1995. Teamwork in group model building. System Dynamics Review 11, no. 2: 113–137. Richardson, G. P, and A. L Pugh. 1981. Introduction to system dynamics modeling with DYNAMO. MIT Press Cambridge, MA, USA. Rosenhead, J. 1996. What's the problem? An introduction to problem structuring methods. Interfaces: 117–131. Rouwette, E., S. Hoppenbrouwers, and B. C. Dangerfield. 2008. Collaborative systems modeling and group model building: a useful combination. In 26th International Conference of the System Dynamics Society. Scaife, Mike, and Yvonne Rogers. 1996. External cognition: how do graphical representations work? International Journal of Human–Computer Studies 45: 185– 213. Shaft, T. M, and I. Vessey. 2006. The role of cognitive fit in the relationship between software comprehension and modification. MIS Quarterly 30, no. 1: 29–55. Tarski, A., and J. Corcoran. 1983. Logic, semantics, metamathematics: papers from 1923 to 1938. Hackett publishing company. Vennix, J. A.M, H. A Akkermans, and E. A.J.A Rouwette. 1996. Group model-building to facilitate organizational change: an exploratory study. System Dynamics Review 12, no. 1: 39–58. Vennix, J. A.M. 1996. Group model building: Facilitating team learning using system

11 dynamics. Wiley. Vessey, I. 1991. Cognitive Fit: A Theory-Based Analysis of the Graphs Versus Tables Literature*. Decision Sciences 22, no. 2: 219–240. Vessey, I., and D. Galletta. 1991. Cognitive fit: An empirical study of information acquisition. Information Systems Research 2, no. 1: 63–84. Wegmann, A., L. S. Lê, G. Regev, and B. Wood. 2007. Enterprise modeling using the foundation concepts of the RM-ODP ISO/ITU standard. Information Systems and E- Business Management 5, no. 4: 397-413. Wegmann, A., G. Regev, J. D de la Cruz, L. S Lê, and I. Rychkova. 2007. Teaching Enterprise Architecture And Service-oriented Architecture in Practice. In Workshop on Trends in Enterprise Architecture Research (TEAR 2007), 13. Wegmann, A., G. Regev, and B. Loison. 2005. Business and IT Alignment with SEAM. In Proceedings of 1st International Workshop on Requirements Engineering for Business Need, and IT Alignment, Paris. Weinberg, G. M. 2001. An introduction to general systems thinking (silver anniversary ed.). Dorset House Publishing Co., Inc. New York, NY, USA. Zhang, J. 1997. The nature of external representations in problem solving. Cognitive science 21, no. 2: 179–217. Zhang, J., and D. A Norman. 1994. Representations in distributed cognitive tasks 1. Cognitive science 18, no. 1: 87–122.

12