Mitigating Cognitive and Neural Biases in Conceptual Design

MITIGATING COGNITIVE AND NEURAL BIASES IN CONCEPTUAL DESIGN

Gregory Matthew Hallihan

A thesis submitted in conformity with the requirements for the degree of Master’s of Applied Science Graduate Department of Mechanical and Industrial Engineering University of Toronto

Gregory Matthew Hallihan

Master’s of Applied Science

Graduate Department of Mechanical and Industrial Engineering University of Toronto

2012

Abstract

Conceptual design is a series of complex cognitive processing tasks and research seeking to further understand design cognition will benefit by considering literature from the field of psychology. This thesis presents two research projects that sought to understand and mitigate design biases in conceptual design, through the application of theories from biological and cognitive psychology. The first of these puts forward a novel model of design creativity based on connectionist theory and a neurological phenomenon known as long-term potentiation. This model is applied to provide new insights into design fixation and develop interventions to assist designers overcome fixation. The second project seeks to establish that cognitive heuristics and biases predictably influence design cognition. Two studies are discussed that examined the role of confirmation bias in design. The first establishes that confirmation bias is present during concept generation; the second demonstrates that decision matrices can mitigate confirmation bias in concept evaluation.

ii Acknowledgements

I would like to thank my supervisor, Professor Li Shu, for her contributions to this research and her mentorship. I would also like to thank Professor Birsen Donmez and Professor Greg Jamieson for their contributions as members of my thesis committee.

I am grateful to my parents, Mike and Terry, who have always supported and encouraged fulfilment through education, and to Laura, for her patience and support.

I would also like to thank Hyunmin, Jay, and Jayesh, for their advice and friendship over the last two years.

The funding for this research was generously provided by the Natural Sciences and Engineering Research Council.

iii TABLE OF CONTENTS

Abstract ...... ii Acknowledgements ...... iii List of Tables ...... vi List of Figures ...... vii List of Appendices ...... viii

THESIS INTRODUCTION ...... 1

CHAPTER 1: Long-Term Potentiation and Design Creativity ...... 3

1. Introduction ...... 3

2. The Biology of Information Processing ...... 4 2.1. Neural Transmission ...... 4 2.2. Long-Term Potentiation ...... 6 2.3. The Biology of Creativity ...... 7

3. Connectionist Theories and Creativity ...... 9 3.1. Information Processing in Connectionist Networks ...... 9 3.2. Network Activation and Associative Creativity ...... 10 3.3. Summary ...... 12

4. The Role of LTP in Design Creativity and Design Fixation ...... 13 4.1. Design Fixation ...... 13 4.2. LTP and Design Fixation ...... 15 4.3. Practical Contributions Regarding Design Fixation...... 17 4.3.1. Incubation and Insight in Design Fixation ...... 17 4.3.2. Enhancing Awareness of Design Fixation ...... 19 4.3.3. Using Physical Activity to Enhance Defixation ...... 23

5. Studying Physical Activity and Defixation ...... 24 5.1. Methods and Procedure ...... 24 5.1.1. Participants ...... 24 5.1.2. Procedure ...... 25 5.1.3. Quantifying Fixation ...... 27 5.2. Results ...... 27 5.2.1. Rater Reliability ...... 28 5.2.2. Physical Activity ...... 28 5.2.3. Education ...... 29 5.2.4. Concept Quantity Among Engineering Students ...... 29 5.3. Discussion ...... 31 5.3.1. The Effect of Physical Activity ...... 31 5.3.2. The Effect of Education ...... 32

iv 5.3.3. Concept Feasibility ...... 34 5.3.4. Quantifying Fixation ...... 35 5.3.5. Concept Quantity ...... 36

6. Summary and Conclusions ...... 37

CHAPTER 2: Cognitive Bias and Confirmation in Design ...... 39

1. Introduction ...... 39

2. Cognitive Heuristics and Biases in Design ...... 40 2.1. Heuristics in Design and Psychology ...... 40 2.1.1. The Use of Cognitive Heuristics in Design ...... 41 2.2. Design Relevant Cognitive Heuristics and Biases ...... 42 2.2.1. Design Relevance ...... 46

3. Confirmation Bias ...... 57

4. Study 1: Confirmation Bias in Concept Generation ...... 47 4.1. Observational Research in Design ...... 48 4.1.1. Verbal Protocols as Observational Data ...... 48 4.1.2. Analyzing Verbal Protocols ...... 49 4.2. Study 1: Methods and Procedure ...... 50 4.2.1. Participants ...... 50 4.2.2. Procedure ...... 50 4.2.3. Qualitative Coding...... 51 4.3. Results ...... 54 4.3.1. Ratio of Confirmation to Disconfirmation ...... 54 4.3.2. Qualitative Observations ...... 55 4.4. Protocol Analysis Limitations ...... 57

5. Study 2: Mitigating Confirmation Bias in Concept Evaluation ...... 58 5.1. Participants ...... 58 5.2. Problem 1 ...... 59 5.2.1. Procedure ...... 59 5.2.2. Results ...... 60 5.2.3. Discussion ...... 61 5.3. Problem 2 ...... 62 5.3.1. Procedure ...... 63 5.3.2. Coding Confirmation and Disconfirmation ...... 64 5.3.3. Results ...... 65 5.3.4. Discussion ...... 67

6. Summary and Conclusions ...... 70

v CHAPTER 3: Thesis Summary ...... 72

Long-Term Potentiation and Design Creativity ...... 72 Cognitive Bias and Confirmation in Design ...... 73 Conclusion ...... 75

REFERENCES ...... 77

vi LIST OF TABLES

Table 1.1 Participant demographics ...... 25

Table 2.1 Design relevant cognitive heuristics and biases ...... 46 Table 2.2 Participant groups and assigned design problems ...... 50 Table 2.3 Number of confirmatory and disconfirmatory cases per group ...... 54 Table 2.4 Participant responses in Problem 1 ...... 60 Table 2.5 Conditions and data for Problem 2 ...... 66

vii LIST OF FIGURES

Figure 1.1 Structure of a neuron and a network of neurons with an enlarged synaptic connection ...... 5 Figure 1.2 Associative hierarchies around the word table (Mednick, 1962) ...... 11 Figure 1.3 Hypothetical associative map based on the FBS ontology ...... 21 Figure 1.4 Relationship between participants’ reaction times when judging stimulus familiarity with the stimulus’ solution frequency ...... 23 Figure 1.5 Example concept shown to participants to induce fixation ...... 25 Figure 1.6 Mean fixation scores before and after defixation by experimental condition ... 28 Figure 1.7 Mean fixation scores before and after defixation by educational background.. 29 Figure 1.8 Engineering students’ mean fixation scores before and after defixation by experimental condition ...... 30 Figure 1.9 Mean number of concepts generated by engineering students before and after defixation by experimental condition ...... 31 Figure 1.10 Concept resembling a circuit diagram from an electrical engineering student . 33 Figure 1.11 Functional decomposition with no unified concept ...... 33

Figure 2.1 Protocol analysis of a confirmatory case ...... 53 Figure 2.2 Ratio of instances of confirmation to disconfirmation by group ...... 54 Figure 2.3 Alternatives in Wason’s (1968) confirmation bias experiment ...... 59 Figure 2.4 Problem 1 alternatives ...... 60 Figure 2.5 Problem 2 evaluation concepts and example concept ...... 63 Figure 2.6 Coded participant matrix ...... 65 Figure 2.7 Coded participant notes ...... 65 Figure 2.8 Confirming and disconfirming instances evaluated between matrix and no matrix conditions ...... 67

viii LIST OF APPENDICES

Appendix A Demographic Questionnaire ...... 89 Appendix B.1 Design Problem Before Defixation Task ...... 90 Appendix B.2 Design Problem After Defixation Task ...... 91 Appendix C.1 Defixation Task Instructions ...... 92 Appendix C.2 Defixation Task Story Material ...... 93 Appendix D Recall Test for Defixation Material ...... 95 Appendix E Task Difficulty Questionnaire ...... 97 Appendix F Fixation Coding Instructions ...... 98 Appendix G Rater’s Raw Scores for Fixation Coding ...... 99 Appendix H.1 Participant Concepts Ranked Low in Fixation ...... 105 Appendix H.2 Participant Concepts Ranked High in Fixation ...... 107 Appendix I Design Problems and Biological Analogies ...... 109 Appendix J Coded Verbal Protocol ...... 111 Appendix K Confirmation and Disconfirmation Coding Scheme ...... 118 Appendix L Confirmation Bias Problem 1 ...... 120 Appendix M Confirmation Bias Problem 2 ...... 122 Appendix N.1 Treatment Group Instructions for Concept Evaluation ...... 124 Appendix N.2 Control Group Instructions for Concept Evaluation ...... 125 Appendix O.1 Example of Participant Concept Evaluation Matrix ...... 126 Appendix O.2 Example of Participant Concept Evaluation Notes ...... 127

ix THESIS INTRODUCTION

Concept generation and evaluation are critical phases of the design process; concept generation affords the opportunity to develop a diverse and novel set of concepts, and concept evaluation determines which of these will be subject to additional development and ultimately production (Dieter, 2000). While conceptual design only accounts for a small proportion of the cost incurred to bring a product to market (~5%), decisions made in this phase determine the majority (70-80%) of the manufactured product cost (National Research Council, 1991).

Equally important is the fact that poor decisions early in the design process can become compounded at later stages and lead to negative process and product outcomes. Therefore, research aimed at enhancing design methods and designer capabilities during concept generation and evaluation significantly benefit product design generally.

Creativity and decision-making are two integral components of successful conceptual design; creativity is required for the generation of novel and viable concepts, and rational decision-making is required to move towards optimal design outcomes. Both creativity and decision-making have been directly studied in the field of psychology long before formal branches of design science took similar interest. Because of this, and the growing desire to better understand design cognition, recent trends in design research reveal an increasing application of literature from psychology to design (e.g., in learning, analogical reasoning, computational design, problem solving, etc.). However, valuable contributions still remain to be made with respect to creativity and decision-making in design. For example, the fields of biological psychology and neuroscience offer insights into the biological mechanisms of cognition, however they remain largely overlooked in the design literature. In addition, the study of cognitive heuristics and biases has greatly contributed to the understanding of human

1 decision-making, but literature pertaining to design biases seems to have developed independently of these insights.

The purpose of this research is to further understand and mitigate obstacles to design creativity and decision-making, by applying research from the fields of biological and cognitive psychology. Practical applications of this research could contribute to enhancing design research paradigms, developing new design tools and methodologies, and informing design education.

The remainder of this thesis is divided into 3 chapters. Chapter 1 discusses the role of

Long-Term Potentiation (LTP) – a mechanism of neural plasticity, in design creativity. A theoretical discussion is presented regarding LTP’s role in Design Fixation – an unintentional adherence to a limited set of concepts or problem strategies during design. This is followed by the discussion of an experiment that was informed by the proposed model, which attempted to alleviate design fixation via the incorporation of physical activity with a defixation exercise.

Chapter 2 discusses the role of cognitive heuristics and biases in design decision- making. The emphasis is placed on understanding and mitigating the negative influence of

Confirmation Bias – a tendency to search for or interpret evidence in such a way as to maintain pre-existing beliefs. Two research projects that sought to empirically evaluate the role of confirmation bias in concept generation and evaluation are presented. This research incorporated naturalistic observation and qualitative analysis, as well as controlled experimentation and quantitative analysis. The first project examined verbal protocols collected from an observational study, in order to identify whether or not confirmation bias was present during concept generation. The second project was an empirical study examining the effect of using formalized decision matrices to mitigate confirmation bias during concept evaluation.

Finally, Chapter 3 summarizes this research in its entirety. While two distinct research projects are presented, the unifying theme throughout is to understand designer cognition from a psychological perspective to enhance design creativity and decision-making. 2 CHAPTER 1 LONG-TERM POTENTIATION AND DESIGN CREATIVITY

1. INTRODUCTION

Creative thought is the process by which individuals and groups advance society, whether through incremental changes to existing technologies, or through innovations that change the perceived limitations of systems. Developing a better understanding of the cognitive mechanisms of creativity supports concept generation in the design process (e.g., through development of more effective tools and methods for innovation [Cagan, 2007]). While it has been argued that any credible understanding of creativity must be consistent with a modern understanding of brain function (Pfenninger & Shubik, 2001), researchers have only just begun to examine the biological foundations of creativity and the implications this could have for designers. This chapter will discuss the role of long-term potentiation (LTP) in design; this neuro-biological phenomenon has been implicated in learning and memory development but is under investigated with respect to creativity.

First, a theoretical explanation of creativity emphasizing LTP’s role in modifying neural networks will be presented. Connectionist and associative theories of creativity are discussed, as they serve as a foundation for understanding creativity in complex information-processing networks. This theoretical discussion then turns to the role of LTP in design fixation, which is a well-studied phenomenon that has been demonstrated to inhibit creativity during conceptual design. Finally, the results of a study that arose from considering LTP’s role in creativity (the effect of physical activity on fixation) are discussed. The goals of this research are to: 1)

Establish that a theoretical-neurological model based on long-term potentiation can contribute to

3 better understanding design creativity, 2) describe how this model could be applied to enhance creativity in design, and 3) test a practical application of this model to mitigate design fixation.

2. THE BIOLOGY OF INFORMATION PROCESSING

The currently accepted view of information transmission in the brain can be largely attributed to the work of Nobel laureate (1906) Santiago Ramon y Cajal. Cajal was a principle proponent of the Neuron Doctrine, which states that neurons are discrete units responsible for the processing of information in the brain (Andres-Barquin, 2002). Over the past century the understanding of the central nervous system has been advanced, and there have been moderate revisions to the neuron doctrine (e.g., signal transmission between neurons is more complex then the unidirectional transmission of impulses between adjacent neurons, [Bullock et al.,

2005]). However, for the purposes of this research to understand the importance of LTP during creative cognition, the following discussion is limited to the fundamental established principles of neural transmission.

2.1. Neural Transmission

A basic understanding of how the human brain transmits information is needed before beginning to understand how LTP could influence creativity. The following is largely adapted from the discussion of neural transmission by Breedlove et al. (2007a).

The brain is composed of billions of interconnected processing units called neurons, which consist of a cell body, axons, and dendrites (Figure 1.1). Neurons are the basic cellular unit of the nervous system, and transmit information through electric and chemical signals.

Within a neuron, an electric signal transmitted is called an action potential. The action potential is generated when a neuron’s membrane potential is altered in a process called depolarization.

Depolarization results from the flow of ions in and out of the cell body, and if the resultant

4 membrane potential of the neuron exceeds its threshold level (~ -55 mV) an action potential is generated. The action potential is ionically propagated from the cell body, down the axon (an extension of the cell body) to the axon terminals. Axon terminals are structures that form synaptic connections with other neurons, and contain synaptic vesicles with various neurotransmitters (molecules that chemically signal changes in neurons). The arrival of the action potential at the axon terminals triggers the release of neurotransmitters, which diffuse into the synapse, the gap between one neuron’s axon and another’s dendrites, where they bond with receptors on the post-synaptic neuron (Figure 1.1). Dendrites are also extensions of the neuron’s cell body, but unlike axons are specialised to receive signals from other neurons. Receptors are structures on dendrites that bind with specific neurotransmitters, and produce specific responses in the post-synaptic neuron upon successful binding. Based on the receptor characteristics of the post-synaptic neuron, and the neurotransmitters released by the pre-synaptic neuron, the post- synaptic neuron may or may not reach the threshold level of depolarization required to generate an action potential and continue propagating the signal. Neuron physiology is variable, and there are multiple types of neurons and neurotransmitters, however this introductory explanation is sufficient to understand the role of LTP in information processing.

FIGURE 1.1. Structure of a neuron (Left) adapted from Breedlove et al. (2007a) and a network of neurons with an enlarged synaptic connection (Right) adapted from Young (2007).

5 2.2. Long-Term Potentiation

Bliss et al. (2003) provide the following description of LTP:

In long-term potentiation, the strength of synapses between neurons in the central nervous system is potentiated for prolonged periods following brief but intense synaptic activation (pp. 607).

This description defines LTP as a mechanism responsible for increasing the likelihood that activation from a pre-synaptic neuron will lead to activation in a post-synaptic neuron following the co-activation of those neurons. If the synapse is the point of information transmission between neurons, then LTP is responsible for an increase in the efficiency or likelihood that information/signals originating at a pre-synaptic neuron will successfully propagate in the post- synaptic neuron. Short-term potentiation (STP) refers to more rapidly occurring potentiation that subsides quickly (i.e., 5-20 minutes [Malenka & Nicoll, 1999]) as opposed to LTP, which can last from 30 minutes (Bliss & Lomo, 1973) to months (Barnes, 1979). Since the effects of LTP and STP on neural connectivity are similar with respect to how they influence the spread of activation in a neural network, this discussion uses the term LTP generally to describe neural potentiation.

The first evidence for LTP was provided by Bliss and Lomo (1973), as well as Bliss and

Gardner-Medwin (1973), who observed long-lasting changes in the neurons of rabbits after externally stimulating large groups of neurons. Their research demonstrated that the external stimulation of neurons could result in an increase in their synaptic efficiency, even after the stimulation was removed. Further research revealed that LTP was not limited to instances in which neurons were stimulated artificially. Thompson et al. (1983) demonstrated that LTP occurred in rabbits induced to exhibit a conditioned eye-blink response. The researchers were able to electrophysiologically measure a change in synaptic efficiency between neurons in response to the rabbits’ behavioural conditioning. Additional examples of the documentation of

LTP in response to behavioural conditioning can be found in Teyler and DiScenna (1987). 6 Breedlove et al. (2007b) provide an overview of some of the possible mechanisms through which LTP occurs, including alterations in neurotransmitter release, neuron receptor characteristics, synapse size, or enzymes that modulate neurotransmitters. LTP may also be the result of changes to neuronal structure, such as an increased proliferation of dendrites and subsequently an increased number of synapses between neurons.

Because LTP results from behavioural responses to external stimuli, it is not surprising that it has been proposed as a mechanism for memory formation (Bliss & Collingridge, 1993) and learning (Martinez & Derrick, 1996; Cline, 1998; Van-Praag et al., 1999). In a distributed memory system, information is stored in a network of neurons through the connections and changes in synaptic function between those neurons (Martinez & Derrick, 1996). Therefore, the connections between neurons and the patterns of transmission between them are responsible for the encoding and storage of memories. For new memories to be formed, the network as it exists needs to alter the communicative distribution of activation, which is precisely what LTP achieves. For a comprehensive explanation of the role of LTP in memory refer to Martin et al.

(2000). Most importantly, LTP is a mechanism through which the connections in a biological information-processing network are altered, which has direct implications for how such a network can “think” creatively.

2.3. The Biology of Creativity

Given that LTP has been proposed as a mechanism of learning and memory, it is reasonable to assume that it is also involved in creativity. However, although psychologists have studied creativity as a distinct cognitive process since the early 1900’s (MacDougall, 1905;

Perky, 1910), Jung et al. (2009) argue that, until recently, there has been little advancement in developing a neuro-biological explanation of creativity. This section will discuss the existing biological theories of creativity in addition to the proposed theory based on LTP.

7 The most well known theories involving neural correlates of creativity are likely those involving hemispheric specificity. These theories assume that the two cerebral hemispheres have functionally distinct roles; the right hemisphere of the brain is relied on for creative cognition and the left is relied on for analytic cognition (Martindale, 1999; Heilman et al., 2003).

However, Dietrich (2004) has suggested that specific neural circuits, independent of hemispheric specialization, are responsible for different “types” of creativity. Dietrich’s argument is based on research indicating that different levels of cognitive regulation from pre- frontal cortical areas dictate which neural circuits are relied on for information processing. Even more recently Gabora (2010) proposed a neurological model of creativity in which “atypical” neural structures are activated during creative thought; this activation supposedly allows individuals to form new neural connections resulting in novel associations. Gabora’s theory focuses on the role of connectivity between neurons in creative cognition, which highlights the importance of understanding how that connectivity changes.

There has been little discussion on how mechanisms of neural plasticity such as LTP influence creativity. Although Lippin (2001) credited Greenberg, an evolutionary biologist, for supporting LTP as a possible biological mechanism involved in creativity, no detailed explanation is provided or published. Others have discussed the influence of neural plasticity on creativity more generally (Haier, 1993; Heilman et al., 2003), but do not directly implicate LTP.

Yet, as mentioned, LTP has been proposed as a mechanism of memory formation (Bliss &

Collingridge, 1993) and learning (Martinez & Derrick, 1996; Van-Praag et al., 1999); this in combination with the fact that LTP moderates the connection strength between neurons suggests that it could be involved in creative cognition as well. However, to fully appreciate the implications of changing neural connectivity in creative cognition it is useful to consider connectionist theories on the subject.

8 3. CONNECTIONIST THEORIES AND CREATIVITY

Connectionist theories are aimed at understanding and modelling cognition through interconnected networks of simple units. Martindale (1995) provides two reasons to consider connectionist models when developing cognitive theories: 1) these models unify multiple psychological theories, and 2) directly relevant to the role of LTP in creativity, connectionist models parallel models based on biological information-processing networks. Connectionist theories also have practical applications in design. For instance, design techniques that encourage distributed thought processes (e.g., brainstorming [Osborn, 1963], or the use of random stimuli [de Bono, 1979]), are fundamentally tied to connectionist theories of associative creativity (Mednick, 1962). This section provides an introduction to connectionist theory, in order to demonstrate the clear link between LTP and creative cognition.

3.1. Information Processing in Connectionist Networks

Fodor and Pylyshyn (1988) describe connectionist systems as networks comprised of simple but highly interconnected processing units. The terminology for these units varies, but will be consistently referred to here as “nodes.” Fodor and Pylyshyn further note there are various levels of connection strength between nodes in a network, and the connection strengths and input activation at each node determine how information is transmitted through the system.

Parallel distributed processing theory (Rumelhart et al., 1986), which also describes the spread of activation in connectionist networks, has been largely integrated into the modern connectionist perspective and will not be discussed independently.

Spreading activation theory describes a model for searching connectionist networks

(Quillian, 1962; Collins & Loftus, 1975) and can be used to predict the likelihood that specific nodes will be co-activated. As a network is searched activation spreads from a starting point outwards, directed by the connection strength between nodes. Activation spreads out along strong connections and is resisted at weak connections. Nodal connection strength therefore 9 dictates the spread of activation in the network and the frequency with which specific nodes are co-activated after some input stimulation. In a biological connectionist network attentional processes also dictate the spread of network activation. It is generally agreed that less focused attention allows for a wider distribution of activation, whereas focused attention leads to the activation of fewer nodes (Mendelsohn, 1976; Sternberg & Lubart, 1999). Dietrich (2004) has also proposed that when attention is unfocused, the processing of information occurs in neural circuits that are not regulated by conscious thought.

The fact that the distribution of activation in a network is dictated in part by nodal connectivity highlights the importance of mechanisms that can alter this connectivity. Donald

Hebb (1949) was an early, if not the original, proponent of connectionist theory. Even before

LTP was documented he proposed that neurons that fired together would become more efficient information processing units. For example, if an input entering the system triggered node A to fire, and node A firing caused node B to fire, the connection between nodes A and B would become more efficient as B fires more “easily” in response to activation from A. While this was originally a theoretical proposal, research demonstrating LTP provided the mechanism through which this change occurred in the brain.

Flexibility is critically important in developing a biological model of design creativity because, as Helms and Goel (2012) argue, in conceptual design the design problem and solution co-evolve. Designers must integrate and process information in novel ways if they are to generate concepts that are truly innovative. This necessity to formulate new concepts is facilitated by the ability of a flexible network to form new associations between nodes due to changing connection strengths.

3.2. Network Activation and Associative Creativity

Mednick (1962) proposed a model of creativity based on associative hierarchies (which can be considered as generalized networks) composed of various nodes that represent an 10 individual’s knowledge. The activation of any one node will, by association, lead to the activation of others in the hierarchy. Mednick (1962) proposed that the associative strengths for certain words or ideas would be stronger than others (e.g., “Table – Chair” vs. “Table – Food”), and dictate the “slope” of an associative hierarchy. Steep slopes indicate strong associations between relatively few concepts, whereas flat slopes indicate weaker associations between many concepts (see Figure 1.2). Mednick proposed that the probability of developing creative solutions was proportional to the number of associations made; therefore creativity becomes dependent on how widely distributed activation is in the network, which is dependent on the connection strengths between nodes.

14 Steep Associative Hierarchy 12 Flat Associative 10 Hierarchy

2 Associative Response Srength 0 Chair Cloth Wood Leg Food Mable

FIGURE 1.2. Associative hierarchies around the word table (Mednick, 1962).

Despite Mednick’s (1962) belief that a more distributed spread of activation (or flatter associative hierarchies) contributed to more creative ideas, diversity of associations alone is unlikely to lead to enhanced creativity in the design context. In engineering design, and to a varying degree in psychological research (see Dietrich, 2004; Dijksterhuis & Meurs, 2006 for an example of the variability), creative ideas must be novel and feasible given established problem constraints. One can imagine that a very distributed spread of activation could lead to the co- activation of typically un-associated nodes, which in turn could contribute to extremely novel

11 associations. However, having completely unrelated associations does not ensure that the resulting concept is feasible.

To account for this, more recent research has focused on the role of graded activation distribution within a connectionist network. Gabora (2010) proposed that widely distributed activation in a network is necessary to develop concept originality, but more restricted or focused activation in the network is necessary for developing concept feasibility. This proposal seems reasonable if it is accepted that the novelty of an idea is positively correlated with the number of associations formed in an information-processing network. However, Gabora provides another explanation based on the frequency with which certain nodes are co-activated, and suggests that a widely distributed spread of activation is more likely to lead to associations between nodes that are weakly connected; these nodes would therefore have a lower probability of being co-activated. In this model, creativity is not dependent on the number of associations made but on the co-activation of weakly connected nodes leading to novel associations.

3.3. Summary

The spread of activation within connectionist networks provides an explanation for the generation of creative thoughts. Strongly connected nodes will be frequently co-activated, which may be required to develop the appropriateness or feasibility of concepts, however these strongly connected nodes are less likely to contribute to the development of novel associations and original ideas. Widely distributed activation, or the distribution of activation between typically un-associated nodes, is likely required to generate novel concepts. Most importantly, connection strengths in a biological information-processing network are altered by LTP. One possible implication of this is that as the connection strength between one set of nodes is enhanced, it becomes probabilistically less likely that activation will spread out along alternate connections and lead to novel associations.

12 4. THE ROLE OF LTP IN DESIGN CREATIVITY AND DESIGN FIXATION

Alterations in the synaptic connection strength between neurons alter the spread of activation between them. The spread of activation in a neural network and subsequent nodal associations may influence creative cognition. The following section explores how LTP could influence creativity in the design process. Specifically, the discussion focuses on design fixation, as this phenomenon is a known obstacle to creativity during concept generation and generally manifests in designers as a restricted diversity in associations and concepts.

4.1. Design Fixation

Generally defined, design fixation is “a blind, and sometimes counterproductive, adherence to a limited set of ideas in the design process” (Jansson & Smith, 1991). However, fixation can be discussed in more specific terms. Cardoso and Badke-Schaub (2011) refer to three types of fixation: 1) Design fixation - the “reuse of previously seen features or principles,”

2) Mechanised thought – the adherence to a constant frame of thought (indicating designers are essentially fixated on a problem-solving strategy), and 3) Memory blocking – fixation resulting from “a cognitive obstruction during memory retrieval.” The present research is primarily interested in the first category of fixation (i.e., the reuse of previously seen features and principles) however mechanised thought is also discussed in terms of strategy adherence.

Fixation has been a phenomenon of interest in psychology since at least the 1940s, when

Duncker (1945) performed an experiment demonstrating that the initially perceived function of an object unconsciously influences its subsequently perceived uses. Duncker presented participants with: a box, tacks, and a candle; and asked them to affix the candle to the wall. In one condition the tacks were presented in the box, while in the other condition the tacks and the box were presented separately. In the latter condition, participants were more likely to use the box as a platform to support the candle (i.e., tack the box to the wall and place the candle on the box) than participants in the former condition, who were more likely to attempt and tack the 13 candle directly to the wall. This specific phenomenon is often referred to as functional fixedness, defined by German and Barrett (2005) as a difficulty considering an item for a function other than its most typical one.

Fixation is also often induced by exposure to stimuli presented during the concept generation process, and can lead individuals to incorporate elements of those stimuli into their own concepts (Jansson and Smith, 1991; Linsey et al., 2010; Perttula and Liikkanen, 2006).

Individuals may also become fixated on strategies they utilize (Simon, 1966; Cardoso & Badke-

Schaub, 2011), which can lead to ineffective or unnecessarily restricted problem solving.

With regards to the influence of fixation on design outcomes, Jansson and Smith’s

(1991) definition implies that it is not always counterproductive. An individual could be fixated on a strategy that leads to a creative solution or positive design outcome. Therefore, assessing the benefit or detriment of fixation is largely based on the outcome of the design process.

However, the goal of concept generation is typically to develop numerous ideas (Dieter, 2000) that can be subsequently evaluated to determine which ones should be refined and selected for development. Therefore, the influence of fixation is more likely to be detrimental when design outcomes are dependent on the consideration of a diverse set of alternatives. In addition, because design fixation can arise from exposure to pre-existing design solutions, it often inhibits concept novelty.

Further complicating the effects of fixation, individuals are typically unaware that they are fixated (Marsh et al., 1999), which makes it more difficult for them to mitigate the possible influence of fixation; even experienced designers have been shown to fixate after explicit instruction not to (Linsey et al., 2010). While it may seem surprising that experts exhibit this behaviour, Chase and Simon (1973) demonstrate that experts often have difficulty processing information from a perspective incongruent with their domain of expertise. This could be a result of experts’ acquisition of a large body of knowledge related to their domain, which 14 reflects the development of a neural architecture specialized in efficiently processing domain specific information. Mednick (1962) suggested that experts have steep associative hierarchies

(with respect to associations involving their field of expertise), which may in turn make it difficult for them to form unconventional associations.

Finally, fixation is a universal phenomenon. For example, in an experiment by German and Barrett (2005) members of an un-industrialized Ecuadorian tribe were primed with the function of completely novel objects, and then asked to solve problems using those objects. The researchers found that priming individuals with the function of an item, even when they had never encountered it before, heavily influenced subsequent use of that item in a fashion congruent with the primed use (relative to a control group that was not primed with the function during problem presentation). This suggests that fixation has a biological foundation and is not solely a result of socio-cultural learning or current educational paradigms. What remains is to compellingly establish that LTP is at the root of fixation’s biological origins.

4.2. LTP and Design Fixation

From a connectionist perspective, fixation can be considered as the inability for activation to spread to nodes other than those associated with the source of fixation. This view is consistent with Ward’s (1994; 1995) theory of Structured Imagination and Path of Least

Resistance. Ward proposed that during the generation of novel ideas, central attributes of novel concepts are determined based on the frequency they are associated with known representations of similar concepts.

Groups of neurons that are frequently activated together develop stronger, more efficient connections due to LTP. As a result, the spread of activation to nodes infrequently associated with the fixation target becomes less likely. For example, in functional fixedness, repeated use of an object for a specific function reinforces connections relating the concept of the object to its typical use. This simultaneously makes it less likely activation will spread to a set of nodes 15 associated with atypical uses of the object. Nodes in this context must embody mental constructs, such as functions, and are best represented by complex assemblages of neurons and the patterns of connections between them (as opposed to individual neurons).

LTP could also be responsible for fixation arising from brief exposure to stimuli, such as example design solutions. Bliss et al. (2003) reported that even the brief stimulation of neurons could lead to LTP, and Malenka and Nicoll (1999) asserted potentiation could be triggered in seconds. If this is the case, LTP could be responsible for fixation even when it results from brief exposure to fixating stimuli. The presentation of an example solution activates neural pathways linked to nodes in the brain responsible for internal representations of the stimuli, enhancing the efficiency through which those pathways communicate information.

Fixation is not permanent. The discussion to this point may appear to suggest that LTP could cause individuals to become permanently fixated as network connections become progressively more efficient. However, this is not the case in reality and is not incongruent with a theory of fixation involving LTP. First, as previously discussed, although LTP leads to long lasting changes (with an upper limit that remains unknown) the duration is variable and typically subsides over time (Bliss et al., 2003). In addition, there are biological mechanisms that act in opposition to LTP, one of these being long-term depression (LTD). Long-term depression is neurological mechanism that reduces the efficiency of synaptic connections and may normalize synaptic weights (Derrick & Martinez, 1995). Therefore, while biological mechanisms of neural plasticity may contribute to fixation, they are not so long lasting or dominant as to lead to permanent fixation. In addition, individuals possess the ability to consciously moderate the focus of their attention, and are constantly bombarded with a variety of stimuli that would lead to the activation of different nodes. If the spread of activation in a connectionist network originates from a point of stimulation, as the diversity of the input increases so too must the distribution of activation. Therefore, the distribution of activity in the network may be 16 intentionally or unintentionally moderated based on the input. Considering these factors, as well as the fact that there are billions of connections in the brain, it seems probabilistically unlikely that any one group of connections could become permanently dominant.

4.3. Practical Contributions Regarding Design Fixation

The discussion to this point has focused on the theoretical role of LTP in design cognition and fixation from a connectionist perspective. However, this theory has the potential to make practical contributions to design theory and methodology. The following section discusses how a cognitive model that incorporates LTP can be useful with respect to better understanding and possibly mitigating design fixation. This will be followed by the discussion of an experiment that sought to empirically test one of these derived applications.

4.3.1. Incubation and insight in design fixation. Incubation and insight are often sequential phases in the ideation process; incubation is a period during which an individual stops consciously working on a problem, after which moments of insight (i.e., enhanced problem solving or creativity) occur (Wallas, 1926). In many instances, insight reflects the spontaneous relief of fixation, as individuals are able to find a solution for a problem that was previously obstructed by some unnecessary adherence to a problem strategy or concept. Incubation and subsequent spontaneous moments of insight are well documented in anecdotal reports (e.g.,

Kekulé’s insight regarding the ring-shaped structure of benzene following a daydream of a snake chasing its tail) as well as scientific evidence (e.g., Wagner et al. [2004] empirically demonstrated that sleep could inspire insight and enhanced problem solving).

While there is strong support for the relationship between incubation and insight, debate remains over the mechanisms responsible for this phenomenon. A popular belief is that insight following incubation is the result of unconscious cognitive processing. For example, Dietrich

(2004) suggested that creativity arising from “spontaneous generation,” (e.g., daydreaming) results from the down-regulation of executive control in pre-frontal cortical areas, allowing for 17 information processing using different neural circuitry associated with more divergent or less constrained thinking. However, some researchers disagree that the unconscious mind processes information in parallel with the conscious to develop new insights.

Smith and Blankenship (1989) suggest that forgetting sources of fixation, and not unconscious processing, is likely responsible for insight following incubation. Similarly, Simon

(1966) argues that incubation allows individuals to forget problem strategies they previously used to interpret problem-relevant information, once these strategies are forgotten new ones can be developed or referenced leading to moments of insight. These perspectives hinge on the assumption that forgetting obstructing information (defixating) facilitates insight. However, because unconscious processing may utilize neural circuitry separate from conscious processing, it is possible that both unconscious processing and forgetting fixation play a role in insight.

Interestingly, incorporating LTP into a theory of design cognition can inform this debate.

LTP is a mechanism responsible for the formation of memories and is therefore relevant to the forgetting fixation hypothesis advocated by Simon (1966) and Smith and Blankenship

(1989). Adopting connectionist terminology, as network connections related to inefficient problem strategies or information weaken or become less efficient, subsequent insight may result from the spreading of activation to previously un-associated nodes in the network. When individuals are not consciously directing attention to the problem during the incubation phase, potentiation between previously activated nodes may subside. This proposal is congruent with those empirical observations regarding the impermanence of LTP (Malenka & Nicoll, 1999;

Bliss et al., 2003). Alternatively, mechanisms such as LTD could be responsible for altering synaptic weights. Therefore, if LTP were responsible for fixation, it would also lend support for the forgetting fixation hypothesis (however it does not refute unconscious processing).

Whether or not insight following incubation is the result of forgetting fixation or unconscious processing has implications for designers. If the unconscious processes information 18 in parallel, to offer insights up to the conscious, designers simply need to stop consciously thinking about the problem and wait for subsequent creative insights. However, if forgetting fixation is responsible, designers can pursue insight more actively. For example, forgetting information works because it allows for the adoption of a new design strategy, therefore designers may attain insights through actively discussing design problems with peers, in order to gain a new perspective. Perhaps the most interesting application would be to help designers detect when they are fixated, and subsequently facilitate the adoption of new strategies to encourage more creative conceptual design.

4.3.2. Enhancing awareness of design fixation. A first step towards mitigating design fixation (or fostering insight) is to help designers detect when they have become fixated on non-optimal design strategies and concepts. A seemingly obvious approach in the current context would be to directly measure LTP or neural activity (if the hypothetical relationship between LTP and fixation were accepted). However, it seems unlikely that imaging techniques used to detect patterns of neural activity, such as fMRI, could be practically incorporated into the design process. While there are areas in the brain that have been shown to be responsible for processing well learned information (e.g., the Fusiform Gyrus and Fusiform Face Area are specialized in processing visually familiar information and developing category expertise

[Gauthier et al., 1999]), limitations in the resolution of imaging systems to detect LTP, in addition to the cost and unwieldy nature of these devices, prevent this from being a practical solution. Therefore directly measuring neurological changes or LTP in order to enhance designer awareness of fixation is not currently a feasible solution. However, since repeated activation of neural connections leads to LTP, and LTP is implicated in memory, more practical methods to detect fixation could involve the use of memory metrics.

Associative maps. It may be possible to detect fixation by having designers generate associative maps, which could be construed as rudimentary externalizations of an internal neural 19 network. If an individual generates one externalization and then attempts to regenerate it at a later time, identifying overlap between the two externalizations may reflect sources of fixation.

Conversely, asymmetries between the two externalizations (e.g., forgotten nodes or new connections) may indicate information that is poorly remembered and less likely to be a source of fixation, which would be useful to expand on. Applying connectionist theory, externalized maps could take the form of a network of nodes.

The content of these maps, including the information encapsulated in a node and how connections are formed, would likely be subject to the desire of the designer and the nature of the design task. However, existing design ontologies may be useful to inform the content. For example, the content of the nodes could be dictated by Gero’s (1990) function-behaviour- structure (FBS) framework. Functions describe what the object is for, Behaviours describe what the object does, and Structures describe components and relations that encapsulate what an object is; functions are derived from behaviours and behaviours from structures (Gero &

Kannengiesser, 2006). A designer could examine the concepts they have generated and use the

FBS framework to decompose concepts in order to identify nodes and connections for associative maps. Figure 1.3 shows the beginnings of such a map, based on a design problem for an automated plant watering system (see Appendix B.1). Other alternatives for the content of these maps could involve externalizing more complex concepts in nodes, such as problem strategies, however more complex approaches would be difficult to standardize.

In the example shown in Figure 1.3, there is only one structure associated with administering water (sprinkler head), however there are two for regulating water flow (timer valve, perforated membrane). This in itself may indicate that the designer is fixated on using a sprinkler head to administer water and should consider other options. Alternatively, if the designer were to generate another map later on, and realize that they were no longer considering

20 the use of a perforated membrane to regulate water flow but only a timer valve, it could indicate that the individual had become fixated on that structure over time.

Timer valve

Regulate water ﬂow

Perforated membrane Water plant

Sprinkler Administer water head

FIGURE 1.3. Hypothetical associative map based on the Gero’s (1990) FBS ontology. (Left to Right: Function, Behaviour, Structure)

Further research is required to determine how to best apply this techniques to detect fixation in design (in addition to whether it is actually effective). Statistical testing could be used to determine the degree of overlap between two externalizations indicative of fixation and not average memory performance. In addition, overlap between any externalizations regardless of the content may not always indicate inefficient problem solving, since fixating on problem- relevant information may lead to successful design solutions. It may also be difficult to standardize the method, which would be useful to validate it experimentally. Analyzing concepts using the FBS ontology is a subjective task, and even researchers experienced with the method do not exhibit perfect reliability. For example Gero and McNeill (2006) used the FBS ontology to analyze design protocols, and reported discrepancies in the FBS coding performed by one rater between design protocols. There is also the risk that repeatedly externalizing fixated material could further reinforce fixation, if it encourages recall for fixating information that had the potential to be forgotten. However, if little progress is being made this method has potential to be used to identify areas of fixation. The process also involves taking a break from actively

21 working on the problem (in the time between when the two maps are created), which may foster insight through incubation.

Detecting fixation using recall speed or accuracy. An alternative method to use memory to detect fixation, which may be easier to experimentally validate, was inspired by psychological methodologies. While participants’ subjective reports of confidence can be used as a metric of memory strength, these reports do not always match objective measures of memory (Qin et al., 2011). Therefore, objective measures such as recall accuracy and speed of recall, may more accurately reflect actual memory strength (and synaptic efficiency) than subjective measures. Presumably information that is recalled more accurately or more quickly is associated with stronger synaptic connections in areas responsible for encoding that information.

The relationship between fixation and memory could therefore be examined by correlating participants’ recall for specific design features or strategies with the degree of fixation they exhibit for those same features.

This hypothesis was examined in a pilot study that will be partially discussed.

Participants were shown a set of pictorial and verbal stimuli representing structures that could be incorporated into solutions for a design problem that they would later solve. After being exposed to the stimuli list, they were given the design problem and asked to generate multiple solutions. They were then given a forced reaction time test for the same set of stimuli they had been previously exposed to. It was hypothesized that the strength of fixation (i.e., frequency stimuli were incorporated into design solutions) would negatively correlate with reaction times.

This experiment proved difficult to control experimentally. While negative correlations were observed between participants’ reaction time to stimuli (r = -0.86 images, r = -0.87 words, n = 8) and the frequency those stimuli were incorporated into their design concepts (see Figure

1.4.), participants’ interpretations of the stimuli list varied greatly. This made it difficult to accurately interpret participant behaviour and responses. Essentially, participants brought an 22 outside knowledge base to the design problem that could not be incorporated into the recall test, but was relied on to complete the design task. It was also unclear if memory strength for design structures induced fixation, or repeatedly referencing a design structure (fixating on it) improved performance on the reaction time task. Therefore, this research program did not proceed beyond the phase of a pilot study.

500 450 400 350 300 250 200 Images 150 Words

Reaction Time (ms) Reaction Time 100 50 0 0 1 2 3 4 Number of Structure Occurrences

FIGURE 1.4. Relationship between participants’ reaction times when judging stimulus familiarity with the stimulus’ solution frequency (use in design solutions).

4.3.3. Using physical activity to enhance defixation. In a review of the effects of physical activity, specifically aerobic exercise, on cognitive function, Kramer et al. (2006) found that exercise was associated with enhanced cognitive processes including: planning, working-memory, focused attention, and multi-tasking. Other researchers working with animal models have found that physical activity is associated with improved brain plasticity, decreased neuroatrophy, and enhanced LTP (Van-Praag et al., 1999; Cotman & Berchtold, 2002; Farmer et al., 2004). These results indicate that there is an interaction between physical activity and LTP.

If LTP influences creative cognition and design fixation, it begs the question of how to incorporate physical activity into the design task to provide some positive benefit for designers.

This research assumes that fixation occurs when the network activation has a restricted

23 distribution, preventing the stimulation of nodes un-associated with the source of fixation. If an individual were to engage in a defixation task (i.e., a task promoting the spread of activation between nodes un-associated with the fixated material) and the connections between those un- associated nodes were strengthened, then any subsequent spread of activation within that network should be more likely to spread in a novel pattern, making fixation less likely.

Hypothesis. If physical activity were combined with a defixation task, it could lead to enhanced defixation; LTP would enhance the synaptic efficiency between nodes that are not associated with the fixation stimuli, while simultaneously allowing for the subsiding of LTP between nodes related to the fixation stimuli. This would encourage the spread of activation along new pathways in the neural network. In this case, the defixation task must engage the individual in some activity unrelated to the design task, and could be viewed as a form of active incubation. This hypothesis was tested in the following experiment.

5. STUDYING PHYSICAL ACTIVITY AND DEFIXATION

A study was conducted to examine the relationship between defixation and physical activity. The initial hypothesis was that coupling physical activity with a defixation activity would foster a novel distribution of activation in a neural network, mitigating fixation.

5.1. Methods and Procedure

5.1.1. Participants. Twenty-four University of Toronto students (16 males, 8 females) volunteered to participate in the study. Participation lasted approximately 45 minutes; participants were compensated monetarily ($15) for completion of the study. Table 1.1. displays demographic and experimental information for the participants included in the analysis. The majority of the participants were from the faculty of engineering; the remainder were from other faculties with no one faculty constituting a second majority. The University of Toronto’s Social

24 Sciences, Humanities and Education Research Ethics Board granted approval for this research.

Participants were recruited through poster advertisements placed around the University of Toronto campus and through e-mails sent to individuals known through the research labs participant network. Participants were required to speak English fluently, to have no previous experience speaking or reading Swedish (the defixation activity involved Swedish phrases), and to have no previous experience with the design activity provided.

TABLE 1.1. Participant demographics Gender Mean Age Engineering Condition N Ratio (SD) Students (%) Control 12 5♀ : 7♂ 22.4 (4.9) 7 (58) Treatment 12 3♀ : 9♂ 23.6 (5.9) 9 (75) Total 24 8♀ : 16♂ 23.0 (5.3) 16 (67)

5.1.2. Procedure. After participants read and signed the informed consent form they completed a demographics questionnaire (see Appendix A). They were then given a design problem that they were instructed to generate solutions for (see Appendix B.1); the design problem was adapted from Perttula and Liikkanen (2006). The designer was asked to generate concepts for an automated watering system that would administer 1/10th of a litre of water per week to a potted plant. An example solution for this problem was shown to participants to act as a source of fixation (see Figure 1.5). Participants were given 10 minutes to generate as many concepts that solved the design problem as they could.

A timer valve has been attached to a house water main. At predetermined intervals, the valve opens and allows the desired amount of water to flow through a showerhead onto the plant.

FIGURE 1.5. Example concept shown to participants to induce fixation (image and description from Perttula & Liikkanen, 2006).

25 Defixation. After generating concepts for 10 minutes participants were stopped and instructed to perform the defixation task (see Appendix C.1), which they were told was

“divergent thinking” activity meant to activate the other hemisphere of their brain. The defixation task lasted 10 minutes, and required the participants to read a short story written in

Swedish with English translations (see Appendix C.2). Participants were told that they would be tested on the material at a later point in the study, and were instructed to memorize the content of the story as well as the translations of the Swedish phrases. The defixation task was intended to stop participants from thinking about the design problem.

Physical activity. Participants were randomly assigned to either the control or treatment condition. In the treatment group, participants were required to engage in physical activity in combination with the defixation task. The physical activity required participants to perform aerobic exercise by stepping up and down from an aerobic step-block (6-inches high) at a self- determined rate; participants were asked to keep their rate of stepping comparable to climbing a flight of stairs. Participants in the treatment condition performed the physical activity for the entire 10 minutes that they took to complete the defixation task, the written material was provided attached to a clipboard to allow them to step and read at the same time. In the control condition participants performed the defixation task without physical activity and remained seated at the workstation. This manipulation was the only change between conditions.

Design and recall tasks after defixation. Following the defixation task, participants were given another 10 minutes to generate additional solutions for the design problem. They were given a description of the problem again without the example solution (see Appendix B.2).

After participants completed this second round of concept generation, they were given the recall task to test their performance on the defixation task. Participants were asked to translate 15 phrases used in the Swedish story and answer seven contextual questions (see Appendix D).

Then participants were asked to rate the perceived level of difficulty of each task (i.e., the level 26 of exertion of the aerobic exercise, the difficulty of the language defixation task, the difficulty of the design problem both before and after the defixation task) on a scale of 1 (easy) to 7

(difficult) (see Appendix E). Finally, they were debriefed and paid.

5.1.3. Quantifying fixation. Two independent raters were instructed to evaluate fixation in 123 participant-generated design concepts (see Appendix F). Fixation scores were based on the similarity of each design concept to the example concept. Similarity was rated on a scale from 0.0 - 1.0 (1 indicating high fixation/similarity and 0 indicating no fixation/similarity), along 4 functional dimensions identified by Perttula and Liikkanen (2006): 1) water source, 2) regulation of flow, 3) water transfer, and 4) energy source. A single fixation score for each concept was calculated by averaging the scores across the four functional dimensions, resulting in a concept score between 0 (not-fixated) and 1 (completely fixated). A participant’s fixation score for both before and after the defixation task was calculated by averaging both raters’ concept scores across each concept generated from the respective design period.

The structure of the rating task conforms to Amabile’s (1982; 1996) Consensual

Assessment Technique for evaluating creativity; the raters: 1) had familiarity with the domain,

2) made their assessments independently, and 3) viewed the design solutions in random order.

The design task provided also conforms to guidelines for evaluating creativity outlined by

Amabile in that the task: 1) produces a clearly observable product and 2) allows for flexibility in responses of the designer. Measuring fixation through concept similarity a generally accepted method (Jansson & Smith, 1991; Purcell & Gero, 1996; Linsey et al., 2010)

5.2. Results

The focus of the analysis was to determine if the combination of physical activity with the defixation task had an effect on participant fixation scores relative to the control condition.

However, additional analyses were performed to examine the effect of educational background.

27 5.2.1. Rater reliability. Average fixation scores were used as the metric for comparison between the treatment and control group. Five of the 123 concepts (Numbers: 26,

27, 64, 95, 102) were excluded from analysis due to participant or rater error. The reliability of the fixation measure was assessed using intra-class correlation (ICC). This method is used for cases involving multiple raters evaluating multiple un-ordered observations in order to determine absolute consistency and has been recommended by Shrout and Fliess (1979) for similar rating scenarios. The computed intraclass correlation was statistically significant,

ICC(3,1) = 0.660, F = 5.23, p < .001, and indicated moderate consistency between the raters’ fixation ratings. The raters’ raw scores can be seen in Appendix G. Example concepts scored high and low in fixation, before and after defixation, can be seen in Appendix H.1 and H.2.

5.2.2. Physical activity. A 2X2 (Physical Activity: Yes, No) X (Defixation: Before,

After) mixed-model ANOVA was used to examine the effect of physical activity on fixation scores before and after the defixation activity (see Figure 1.6). There was no significant interaction between defixation and physical activity, F(1,22) = 0.21, p = .65. Fixation scores were not significantly different before or after defixation, F(1,22) = 2.89, p = 0.10. There was no significant effect of physical activity on fixation scores, F(1,22) = 0.03, p = 0.86.

0.7

0.6

0.5

0.4 Before 0.3 After 0.2 Mean Fixation Score 0.1

0 Physical Activity No Physical Activity

FIGURE 1.6. Mean fixation scores before and after defixation by experimental condition.

28 5.2.3. Education. The sample used in this study consisted of 16 engineers (14 males, 2 females) and 8 non-engineers (6 females, 2 males), roughly balanced across conditions (see

Table 1.1). The analysis in section 5.2.2. was repeated including participant education

(Engineer, Non-Engineer) as a covariate (see Figure 1.7). A significant effect of education was observed, F(1,21) = 5.82, p < 0.05. Non-engineering students had lower fixation scores overall

(M = 0.41, SE = 0.04) than engineering students (M = 0.63, SE = 0.05). There was also a significant interaction between education and defixation F(1,21) = 5.00, p < 0.05.

0.8

0.7

0.6

0.5

0.4 Before 0.3 After

0.2 Mean Fixation Score 0.1

0 Engineering Non Engineering

FIGURE 1.7. Mean fixation scores before and after defixation by educational background.

Follow up comparisons were performed using paired samples t-tests to examine the effect of the defixation activity among engineers and non-engineers separately. Non-engineers’ fixation scores were significantly lower, t(8) = 3.38, p = 0.01, after the defixation activity (M =

0.33, SE = 0.07) than before (M = 0.49, SE = 0.06). However, engineers’ defixation scores did not change significantly, t(14) = 0.07, p = 0.95, and were almost the same after the defixation task (M = 0.63, SE = 0.06) as before (M = 0.63, SE 0.06).

5.2.4. Concept quantity among engineering students. The difference between engineering and non-engineering students warranted the examination of the engineers’ data separately. There was no significant difference in fixation scores between the treatment and

29 control conditions, (see Figure 1.8), however cursory examination suggested a difference in the number of solutions generated between groups. Therefore, a 2X2 (Physical Activity: Yes, No) X

(Defixation: Before, After) mixed-model ANOVA was performed comparing the average number of concepts engineering participants generated.

0.9

0.8

0.7

0.6

0.5 Before 0.4 After 0.3

0.2 Mean Fixation Score

0.1

0 Physical Activity No Physical Activity

FIGURE 1.8. Engineering students’ mean fixation scores before and after defixation by experimental condition.

There was no significant effect of the defixation activity F(1,13) = 3.27, p = .09 on the number of solutions generated, however there was a marginally significant effect of physical activity, F(1,13) = 4.52, p = .053 (see Figure 1.9). Engineers in the physical activity condition tended to generate more solutions after the defixation task (M = 3.38, SE = 0.26) than before (M

= 2.75, SE = 0.37). However, engineers in the no physical activity condition tended to generate the same number of solutions after defixation (M = 2.14, SE = 0.34) as before (M = 2.14, SE =

0.34). A paired-samples t-test revealed engineers in the physical activity condition generated significantly more solutions after defixation than before t(7) = 2.38, p < .05.

30 4

2 Before After

1 Concepts Generated (#)

0 Physical Activity No Physical Activity

FIGURE 1.9. Mean number of concepts generated by engineering students before and after defixiaton by experimental condition.

5.3. Discussion

The initial purpose of this experiment was to test the effect of physical activity on the effectiveness of a defixation activity. Follow up analyses revealed potential confounds, and prompted additional analyses accounting for education as a covariate. Interpretations of the observed effects are presented.

5.3.1. The effect of physical activity. There was no significant effect of physical activity on the effectiveness of the defixation task with respect to fixation. One possible reason for this is the limited aerobic exertion and duration of exercise participants in the treatment condition experienced. Participants were asked to step up and down from an aerobic step block, set at 6 inches high, at a brisk and consistent pace. However, physiological measures, such as heart rate, were not recorded and variability in the aerobic exertion actually experienced by participants cannot be accounted for. In addition, studies in which physical activity has been shown to enhance LTP have involved longer and more intense periods of activity. For example,

Van-Praag et al. (1999) observed physical activity enhancing LTP in mice that ran an average of

31 4.78 km per day for 30 days. The physical activity manipulation in this experiment was therefore unlikely to significantly influence LTP.

Another potential factor accounting for the absence of a treatment effect relates to the design task and example concept used to induce fixation. One possible cause of fixation is strategy adherence (Smith & Blankenship, 1989; Simon 1966) (i.e., mechanised thought) and participants may have been more fixated on the strategies they used to solve the problem than the example concept. It seems unlikely that neural connections developed and strengthened over a relatively long period of time (such as problem strategies) would be easily changed by interventions occurring over a relatively short period (the defixation activity).

The lack of an observable effect could therefore be attributed to the fact that neither the treatment (physical activity) nor the defixation activity was likely to encourage participants to forget the way in which they approached the design task before the defixation task. Additional measures (i.e., perceived difficulty of the design task, subjective exertion of the aerobic exercise task, perceived difficulty of the defixation task, and performance on the defixation task) were considered as possible covariates, however none of these variables differed significantly between the treatment and control groups.

5.3.2. The effect of education. There was a clear effect of education on defixation, which necessitated splitting the sample into two distinct groups. This observation is consistent with previous observations, which suggest specific skill differences between individuals contribute to complicating the assessment of creativity (Amabile, 1996). While the engineering students were examined separately, the nine non-engineering participants could not be grouped under any single faculty description.

Engineering students’ fixation scores were quite different than non-engineering students’; engineering students tended to show few signs of defixating whereas non-engineering students appeared to become less fixated after the defixation activity, independent of physical 32 activity. One possible explanation for this effect is that subject matter expertise contributed to fixation in engineering students; qualitative evidence of this was seen in many of these participants’ design concepts. For example, one electrical engineering student sketched concepts that resembled circuit diagrams (see Figure 1.10), another used functional decomposition without actually developing an explicit solution (see Figure 1.11).

Based on these qualitative observations, one factor contributing to engineering participants’ fixation was likely an adherence to strategies they had learned for solving design problems. This is consistent with Purcell and Gero’s (1996) finding that differences in educational approaches between disciplines (mechanical and industrial engineering) can account for differences in the individuals’ degree of fixation while solving design problems.

FIGURE 1.10. Concept resembling a circuit diagram from an engineering student.

FIGURE 1.11. Functional decomposition with no concept by an engineering student.

Another difference between engineering and non-engineering participants was their performance on the language recall task. Non-engineering participants performed significantly

33 better on the recall task than engineering participants. Two possible explanations for this effect are that engineering participants found the task more difficult, or they put less effort into it. The former assumption is supported by a comparison of participants’ subjective ratings of difficulty for the language task. Engineering students’ reported difficulty scores for the language task were significantly higher than those of non-engineering students. This indicates that the two educational groups experienced the defixation activity differently, and demonstrates the importance of accounting for skill differences in the sample population during the development of design experiments.

These educational differences have indirect implications when considering the role of

LTP on fixation. Engineers’ training may increase the strength of neural connections relating to specific solution strategies, and as Simon (1966) argued, incubation effects and forgetting fixation may be a result of forgetting these problem specific strategies. LTP would make it more difficult for activation to spread along alternate paths for individuals whose pre-existing paths are strongly connected through educational training. This finding is consistent with research showing experts have difficulty generating solutions that are incompatible with their domain of expertise (Chase & Simon, 1973). It also suggests that individuals lacking domain-specific expertise relevant to the problem may be easier to defixate.

5.3.3. Concept feasibility. It was briefly considered that engineering students might have been more fixated because they were focused on developing more technically feasible concepts. However, engineering students did develop concepts that were not feasible or ignored the limitations outlined in the problem description (e.g., hiring a gardener to water the plant).

There was no strong evidence to suggest that non-engineering students were more or less likely than engineering students to ignore problem constraints. Therefore, the lower fixation scores observed in non-engineering students were not likely a result of those participants sacrificing feasibility for novelty. In fact, non-engineering participants’ solution feasibility showed no 34 significant evidence of being negatively impacted, relative to engineering students, by a lack of domain specific expertise.

5.3.4. Quantifying fixation. Several concerns arose during data analysis due to the difficulty of quantifying fixation. Although the intraclass correlation was significant, the strength of the correlation (ICC(3,1) = 0.66) was moderate and raters did disagree over the fixation ratings of many concepts. The design task was structured to allow participants flexibility in generating solutions, however many participants’ solutions were ambiguous regarding the functional categories used to rate fixation. For example, participants often indicated that a reservoir would serve as a water source, however they did not specify how the reservoir would be filled (e.g., municipal line or collected rainwater). The raters may therefore score that functional dimension as partially fixated (0.5/1.0) if it is assumed the reservoir is filled from a residential water line (the example source), or unfixated (0.0/1.0) if the rater assumes the water is from a different source. The value of scoring fixation based on these functional categories is dependent on whether or not participants explicitly address functional requirements in their own solutions, which was often not the case.

Assessing fixation often involves evaluating concept similarity relative to the example concept along some objective criterion (e.g., shared design features [Jansson & Smith, 1991;

Purcell & Gero, 1996; Chrysikou & Weisberg, 2005; Perttula & Liikkanen, 2006; Linsey et al.,

2010]). However, other metrics used include originality or novelty (Kurtoglu et al., 2009;

Linsey et al., 2010), concept quantity (Jansson & Smith, 1991; Purcell & Gero, 1996; Linsey et al., 2010), or self-reports in verbal protocols (i.e., explicitly or implicitly evidencing difficulty in developing new ideas [Nicholl & McLellan, 2007; Cheong et al., 2012]). The data collected in this experiment could be reanalysed to determine if participants were fixated on problem solving strategies and not the example provided by considering measures such as concept novelty, or concept similarity between concepts (i.e., concept categories). For example, a participant who 35 repeatedly used solar power and rainwater in their concepts could be said to be fixated on environmental solutions, but because the example concept provided did not utilize these structures, the raters would have scored this participant as not at all fixated. The design community has yet to agree on any single measure of fixation for concept generation studies and challenges to construct validity need further research to address.

With respect to increasing the reliability of fixation ratings, the design task should be structured to eliminate ambiguity regarding the fixation categories. Encouraging participants to focus on developing complete solutions, or discarding ambiguous solutions, would increase consistency. In addition, in discussions with colleagues, and the raters involved in this study, it was determined that detailed training is necessary to ensure that the raters have a shared understanding of fixation. Arbitration between raters could also be used to enhance this understanding (Kan et al., 2010). However, although these approaches could enhance construct validity, they could also introduce systematic individual biases into the rating task.

5.3.5. Concept quantity. There was a statistically significant relationship between physical activity and the number of solutions generated by engineering students. Engineers who performed the defixation activity combined with aerobic exercise generated significantly more concepts after the defixation activity than before, whereas engineering participants who performed the defixation activity without aerobic exercise generated, on average, the same number of concepts before and after the defixation activity. In addition, the subjective reports of task difficulty revealed that individuals in the physical activity condition found it significantly easier to generate solutions after the defixation activity than participants in the control condition.

However, generating more solutions did not equate to generating less fixated solutions.

These results parallel findings regarding the traditional brainstorming technique.

Although brainstorming tends to lead to an increase in the quantity of ideas generated, it does not contribute to increased idea quality (Stein, 1975). In the context of the methodological 36 limitations discussed so far, the reliability of these results can justifiably be viewed with scepticism, in part owing to the number of analyses run and the inflated possibility of committing a Type I error. In addition, based on the limitations discussed in section 5.3.1. it is not being suggested that the effects observed were due to an interaction between physical activity and LTP. However, the trends observed warrant future investigation to better understand how combining physical activity with defixation tasks influences the perceived difficulty of design tasks and how that may relate to solution quantity.

The fact that there was a difference in the number of solutions generated between groups highlights a potential problem with averaging fixation scores across concepts. An important challenge for future fixation studies is to accurately quantify fixation in a way that is not biased by solution quantity. Reinig et al. (2007) identify 4 metrics for assessing the quality of ideation:

1) Number: the number of ideas generated, 2) Sum of Quality: the summed quality of ideas, 3)

Average Quality: the average quality of ideas, or 4) Good Idea Count: the number of ideas generated that exceed a threshold quality. Using reliability analysis, Reinig and colleagues determined that of the four methods, only good idea count was a valid measure of idea quality.

However, they acknowledge that this method ignores the number of poor ideas generated as well as variance in the quality of good ideas. Future fixation studies would benefit by considering these factors, especially, as was the case with the present study, when averaging fixation values across multiple design concepts.

6. SUMMARY AND CONCLUSIONS

Previous design research has sought to explain creativity at different levels of abstraction, from the influence of environmental factors to localizing creativity in the brain.

However, previous research does not explain how biological mechanisms that allow for flexibility in a neural network can influence creativity in conceptual design. The present 37 research began with the assertion that concept generation and evaluation is biased by the efficiency of connections in the brain, and that the efficiency of these connections is subject to change via the neurological mechanism of LTP. LTP offers a possible explanation for how connections between neurons become more efficient, contributing to biased cognition (e.g., design fixation).

Contrary to anecdotal evidence and intuitive expectation, an experimental study did not reveal statistically significant effects of physical exercise on defixation. However, other interesting results were discovered and discussed, such as the role of educational differences on fixation in design. Methodological limitations were identified that could account for why the expected effects were not observed. These limitations gave rise to a discussion of methodological recommendations for future experiments, as well as other ways to re-examine the data collected from this experiment.

This chapter offers a neurological explanation for design creativity and identifies a new research area that can contribute to enhancing concept generation and creativity in design. In addition, the role of LTP in design may become more relevant as techniques of moderating and measuring LTP advance. The desire to understand the neurology of creativity is reason alone to justify further research. However, to truly establish a causal relationship between LTP and creativity in the context of design, studies that can assess neurological function directly are necessary. Even had a strong effect of physical activity been observed in the study presented, it would not definitively prove that LTP plays a causal role in fixation. Therefore, the research focus in the following chapters shifts away from the neurobiological perspective.

38 CHAPTER 2 COGNITIVE BIAS AND CONFIRMATION IN DESIGN

1. INTRODUCTION

Design researchers have established an eclectic body of literature regarding design cognition, with research interests ranging from cognitive science and a theory of design (Gero,

2009) to socio-cultural determinants of creativity (Liu, 2000). Psychological research on cognitive heuristics and biases offers another relevant body of knowledge for application.

Cognitive heuristics are intuitive information-processing strategies that have been shown to, in some instances, contribute to irrational judgments and cognitive biases (Tversky & Kahneman,

1982). Cognitive biases are, from a normative decision-making perspective, inherent judgement errors in human information processing. Design researchers have only recently begun to explore the role of cognitive bias in design (e.g., Viswanathan & Linsey [2010] studied the role of the sunk cost bias in physical prototyping). However, cognitive heuristics and biases have been studied in the field of psychology since at least the 1940’s (Asch, 1946) and can provide useful insights to further understand design cognition and information-processing biases in design.

This chapter first introduces cognitive heuristics and biases, emphasizing their relevance to design. Two studies are then presented that examined the role of confirmation bias – a tendency to seek and interpret evidence in order to confirm existing beliefs, in design. The first study analyzed verbal protocols from a biomimetic-design lab exercise to determine if confirmation bias was present during concept generation. The results suggest that confirmation bias is present during concept generation, and offer additional insights into the influence of confirmation in design. The second study was a controlled experiment examining the effectiveness of decision matrices as tools to mitigate confirmation bias during concept

39 evaluation. While the results indicate that decision matrices can effectively reduce confirmation bias during concept evaluation, possible confounds in the study, as well as limitations to the adoption of formalized decision-making procedures are also discussed.

2. COGNITIVE HEURISITICS AND BIASES IN DESIGN

There are notable differences between the heuristics of interest to psychologists and those commonly discussed in the design literature. The following section discusses these differences, and elaborates on the relevance of cognitive heuristics and biases in design.

2.1. Heuristics in Design and Psychology

Guindon and Curtis (1988) define design heuristics as broadly applied principles “that reduce the complexity of a design problem.” Aronson et al. (2006) define psychological heuristics as “mental shortcuts people use to make judgments quickly and efficiently.” Although heuristics in design and psychology are superficially similar, further comparison reveals fundamental differences in their origin and application.

Heuristics in design are formal rules or procedures deliberately developed for designers to use during the design process. For example, Cormier et al. (2011) developed heuristics for designers seeking to design products that satisfy consumer variation (e.g., a product to be used by a both left and right handed population). Design heuristics are essentially tools that designers can use when the situation is appropriate. Although their use may eventually become less cognitively demanding with practice, the initial acquisition and application are conscious and intentional. Cognitive heuristics differ in that they are not developed for application but are observations of natural occurrences. They are innate information-processing strategies that psychologists have “discovered” humans rely on. In addition, they are relied on without an individual’s conscious intent (Gilovich et al., 2002) and are not explicitly learned. In fact, cognitive heuristics are often discussed in terms of their adaptive benefit from an evolutionary 40 perspective. Because individuals do not consciously apply cognitive heuristics, they are often unaware of how they could lead to cognitive biases and irrational judgments.

2.1.1. The use of cognitive heuristics in design. Cognitive heuristics are used during cognitive processing, under which design is logically subsumed. These phenomena have been shown to influence a diverse set of complex decision-making tasks in interpersonal relationships, medicine, economics, politics, etc. (Gilovich et al., 2002). Given that decision- making is a key component of design (Gero, 1990), design decision-making, at the least, will be subject to a reliance on cognitive heuristics.

In addition to having an opportunity to rely on cognitive heuristics, designers also have an incentive. An innate drive to conserve cognitive effort has been proposed as a hallmark of human information processing (Fiske & Taylor, 1984). This desire is one reason why individuals rely on cognitive heuristics, even when they have an incentive not to (Gilovich et al.,

2002). It is worth noting that design researchers have found evidence indicating designers are also motivated to conserve cognitive effort (Guindon, 1990; Cheong et al., 2012). Therefore, designers may unconsciously rely on cognitive heuristics to minimize cognitive effort, even when they are highly invested in the design task.

There has been little previous research directly investigating the role of cognitive heuristics and biases in design. While Viswanathan and Linsey (2011) have argued that the sunk cost bias (i.e., a tendency to pursue a strategy because of previous investment despite the risk of further losses) contributes to fixation during physical prototyping, this is only one design task and one cognitive bias. Determining how designers use cognitive heuristics and biases requires additional research into each phenomenon individually. This chapter focuses primarily on confirmation bias in concept generation and evaluation, however a discussion of multiple cognitive heuristics and biases that were deemed especially relevant to design will be presented.

41 2.2. Design Relevant Cognitive Heuristics and Biases

Although cognitive heuristics allow for efficient information processing and are generally beneficial, reliance on them may contribute to cognitive biases. A Wikipedia search

(as of May 11, 2012) revealed an impressive number of empirically described cognitive biases:

86 in decision-making, 24 in social judgment, and 51 in memory. However, not all of these are directly relevant to design cognition (e.g., social biases that relate to judgments in interpersonal relationships). The following is a discussion of the heuristics and biases that were thought to be the most relevant to design (see Table 2.1 at the end of Section 2 for a brief summary), for further information on cognitive heuristics see Gilovich et al. (2002).

The Availability Heuristic. Tversky and Kahneman (1973) report that the availability heuristic is relied on when making judgements based on the information that most readily comes to mind. This can lead to biased information processing when the “availability” of information is overly influenced by factors that do not reflect its actual diagnosticity (e.g., overestimating the occurrence of shark attacks because they are highly salient incidents and are thus more available in memory [Plous, 1993]). The availability heuristic can influence simple judgements based on frequency estimates, but has also been shown to influence more complex and serious judgements in real life (e.g., medical decisions involving complex surgery [Gifford–Jones,

1977]). The availability heuristic directly relates to the previous discussion on design fixation; it could easily be one factor accounting for designers’ tendency to rely on well-learned strategies when designing, or to incorporate elements of example solutions (which would be highly available stimuli) into their own designs.

The Representativeness Heuristic. The representativeness heuristic biases judgement by leading individuals to assume that a member of a category is a prototypical representation of that category as whole (Kahneman & Tversky, 1972). This often leads to drawing inaccurate conclusions about large groups from small samples, or small samples from large groups (e.g., 42 stereotyping). This heuristic can also lead individuals to ignore base-rate information. For example, the base rate for a coin coming up heads or tails is p = 0.5. However, after seeing a coin toss come up heads 5 times in a row, most individuals intuitively feel that the next toss has a higher than 50% chance of coming up tails. This is because of a belief that a small sample of tosses should be representative of the outcome of a large number of tosses, even though the base rate for the outcome of each individual toss is still p = 0.5 (Plous, 1993).

The Anchoring Heuristic. Tversky and Kahneman (1982) observed that individuals rely heavily on initial reference points during estimates of frequency or probability. This is referred to as the anchoring or adjustment heuristic; essentially individuals automatically adjust their judgements relative to a reference point that may or may not be relevant. Tversky and

Kahneman illustrate how the anchoring heuristic can lead to biases in evaluating the outcome of compound events. For successful product development, a series of events must occur; even when the individual likelihood of success for each of these events is high, the overall likelihood of each of them occurring can be very low. The anchoring heuristic can lead to overly optimistic estimates for the outcome of conjunctive events, like product design, because the success of an individual event is an anchor that biases the perception of the overall likelihood of success.

The Effort Heuristic. The effort heuristic leads individuals to evaluate alternatives based on the amount of effort that went into developing them, as opposed to relying on more diagnostic evaluation criteria. For example, if individuals believe something took a great effort to develop they will have difficulty disentangling the actual value from this perception of effort

(Kruger et al., 2004). Reliance on this heuristic could lead individuals to make decisions that do not take into account the true value of an alternative.

Sunk Cost Bias. The sunk cost bias refers to a tendency to maintain a course of action due to previous investment (e.g., money, action, time) despite the fact that the prior investment should no longer logically be influencing the decision (Arkes & Blumer, 1985). Viswanathan 43 and Linsey (2011) examined the potential effect of sunk cost bias on fixation, suggesting that the act of building a physical prototype represents an investment, which in turn leads designers to fixate on the current design strategy to avoid a loss of the invested effort.

The sunk cost bias can also lead to more harmful design outcomes than fixation.

Designers and manufacturers may consciously decide to launch products with known design flaws to avoid losses associated with re-designing the product or launching late. While these decisions are usually planned to be cost-optimal they do not always result that way (e.g., the

Ford Pinto’s unsafe fuel tank [Birsch & Fielder, 1994]).

Framing Bias. It has been repeatedly shown that even when given a choice between normatively equivalent outcomes, individuals’ decisions are heavily influenced by how the choice is framed (i.e., whether negative or positive outcomes are emphasized). Tversky and

Kahneman (1981) originally discussed framing as it applied to risky choice problems involving gains or losses. They found that given the choice between normatively equivalent outcomes, individuals strongly tend to be risk averse when dealing with positive outcomes, and risk taking when dealing with negative outcomes. Levin et al. (1998) provide a summary of research demonstrating the influence of framing on risk preference, attribute evaluation, and behaviour adoption, in numerous contexts. Generally, individuals are more likely to act if the action prevents a loss, as opposed to providing a gain. These framing effects can influence behaviours with serious implications. For example, Meyerowitz and Chaiken (1987) found that women were more likely to perform a self-breast exam when informed of the negative consequences of avoiding the exam, than women who were informed of the positive consequences of the exam.

These effects could contribute to design fixation; for example a designer may be less likely to abandon their current course of action if they are focused on the associated gains, instead of the potential losses of committing to a new design strategy.

44 Hindsight Bias. Agans and Shaffer (1994) define the hindsight bias as the “unjustified increase in the perceived probability of an event due to outcome knowledge.” This results in a false sense of confidence when making judgements relating to outcomes that individuals have knowledge of. This confidence is unjustified because without knowing the outcome, the ability to predict it is severely limited. The hindsight bias can lead individuals to discredit others who were unable to predict seemingly obvious outcomes, as well as preventing individuals from learning from past events (Fischhoff, 1975).

Illusory Correlation. Chapman (1967) reports the illusory correlation bias can lead individuals to report correlations between events that are not actually correlated, overestimate correlations, or report correlations in the opposite direction. The illusory correlation bias has been hypothesized to result from the availability or representativeness heuristics (Mullen &

Johnson, 1990). For example, Chapman (1967) presented participants with a series of word pairings and asked them to rate how frequently each pairing occurred. Participants reported that semantically linked words (e.g., bacon-eggs) co-occurred more often than words with no semantic link (e.g., tiger-notebook) even though the number of pairings was equal in all cases.

Chapman and Chapman (1969) also demonstrated that pre-conceived beliefs often lead people to perceive correlations that confirm those beliefs (e.g., personality traits and physical appearance, such as untrustworthy people have tiny eyes).

Topical Mental Accounts. When making judgements involving multiple attributes, people generally have difficulty accurately integrating all the relevant attributes at once.

According to Kahneman and Tversky (1984) only those attributes obviously and directly relevant to the current aspect of focus are considered. Topical mental accounting has been demonstrated to lead to biased purchasing behaviours. For instance, Thaler (1980) found that people would be more willing to exert the additional effort of driving to a different store to save

$5 on a $15 calculator, than to save $5 on a $125 dollar coat; even when they are told they are 45 purchasing the two items together. In this example, individuals only consider the value of savings relative to the item cost, instead of the multi-attribute purchase cost.

2.2.1. Design relevance. These biases and heuristics were discussed because it is believed that they have the potential to influence design cognition. For example, as hypothesized in Chapter 1, and by Hallihan and Shu (2011), the associative strength of design stimuli in an individual’s memory could be predictive of fixation on that stimuli; this could also be explained by a reliance on the availability heuristic. Tversky and Kahneman (1973) point out that while memory works by strengthening connections between events that frequently co-occur, availability works inversely to that, using the “strength of associations as a basis for the judgement of frequency.”

This section is primarily intended as an introduction to the literature and to highlight the heuristics and biases that were seen as particularly relevant to the design. The remainder of this chapter will focus exclusively on the role of confirmation bias in the design process. This decision was partially a result of confirmation bias being described as one of the most prevalent information processing biases (Nickerson, 1998), as well as the difficulty of studying the other phenomena described in a controlled setting. For example, to understand whether or not designers are making decisions/judgements based on the most available information in memory, the researcher must control or ascertain what the most available information in memory is.

TABLE 2.1. Design-relevant cognitive heuristics and biases. Heuristic or Bias Description Availability Making judgements based on the most available information in memory Representativeness A belief that a single instance of a category represents all instances of that category Anchoring Using a baseline stimuli as a reference point for evaluating all other stimuli Effort A belief that the value of something is attached to the amount of effort put into it Sunk Cost Pursuing a strategy because of previous investment, despite the risk of further losses Framing Allowing the frame (positive or negative) of a problem to influence decisions Illusory Correlation Perceiving correlation where none exists Topical Mental Accounts Failing to accurately integrate all attributes of multi-attribute decisions

46 3. CONFIRMATION BIAS

Confirmation bias refers to a tendency to seek out evidence, or interpret evidence in such a way, that is consistent with pre-existing beliefs, at the expense of considering belief- inconsistent information (Nickerson, 1998). A confirmatory bias is evident even when individuals have no vested interest in the belief being evaluated. For example, Koriat et al.

(1980) show that individuals typically attempt to find out if a belief is true, rather than prove that it is false. Nickerson (1998) reports that confirmation bias can lead individuals to fail to use disconfirming evidence to adjust beliefs, accept confirming evidence too easily, misinterpret disconfirming evidence, and fail to consider the diagnostic value of supportive evidence.

Based on recent work (Cheong et al., 2012) it was hypothesized that confirmation bias could be prevalent among novice designers during concept design and contribute to design fixation, (i.e., prevent designers from fully considering the value of alternative design solutions or strategies). Therefore, two studies were conducted in order to better understand the role of confirmation bias in design cognition.

4. STUDY 1: CONFIRMATION BIAS IN CONCEPT GENERATION

The purpose of this first study was to determine whether or not confirmation bias was present during concept generation, and to gain insights into its influence and possible antecedents. The research described in Chapter 1 highlighted methodological difficulties associated with quantifying fixation in the products of controlled concept generation experiments. Therefore, a new methodological paradigm focusing on the naturalistic observation of designers was adopted to examine the influence of confirmation bias in concept generation.

An observational study conducted by Cheong et al. (2012) provided the data collected and analyzed in Study 1. This analysis, also discussed by Hallihan et al. (2012), will be presented following a discussion of the use of observational methods in the study of design cognition. 47 4.1. Observational Research in Design

While experimental studies test the validity of hypotheses or interventions, observational studies are well suited to formulate hypotheses and develop interventions for future experiments. According to Dunbar (1995), an important benefit of an observational study is that it allows researchers to observe more natural and real-world behaviors of participants, whereas those behaviors may be restricted in experimental studies. Observational methods have become increasingly common in design research, and have been previously used to examine: biologically inspired design (Vattam et al., 2008; Helms et al., 2009), iterative design (Adams &

Atman, 1999), collaborative design (Tang & Lee, 2008), and design fixation (Nicholl &

McLellan, 2007). Cross (2001) provides a more detailed discussion of studies in design research that employ an observational paradigm, including the advantages and limitations of this approach; one challenge that will be discussed in more detail is objective data collection.

4.1.1. Verbal protocols as observational data. One method of collecting observational data is to encourage participants to “think aloud”; these dialogues can be recorded and transcribed to generate verbal protocols, which can be analyzed to offer insights into participant cognition (Wickens et al., 2004). Gero (2010) supports the use of protocol analysis in design research, arguing that it has been used extensively as a method to assist researchers in understanding design cognition.

Ericsson and Simon (1993) discuss validity issues that should be considered when using participants’ verbal reports as data (e.g., subjectivity in the coding process), but suggest that instructing participants to think aloud does not significantly alter the cognitive process.

However, Chiu and Shu (2010) observed that the use of a concurrent think-aloud method in design studies can be perceived as unnatural and may place an additional cognitive demand on participants; this can contribute to results that may not reflect real-world performance. As an alternative to actually vocalizing thoughts, designers can be encouraged to participate in design 48 processes naturally and talk aloud as they normally would. While this approach may not capture cognitive mechanisms in as much detail, the process is more natural, may better reflect actual design practices, and is better suited for collaborative design scenarios in which participants contribute simultaneously.

4.1.2. Analyzing verbal protocols. Merriam (2009) recommends that the analysis of protocols should ultimately be tailored towards meeting the needs of the researcher. One of the most common methods of analyzing protocols in psychological and design research, and the most relevant to this research, is qualitative coding.

Qualitative coding involves segmenting a protocol based on categories of interest to the researcher, such as individual statements (Ericsson & Simon, 1993). Those segments are then analyzed based on a predetermined coding structure. Miles and Huberman (1994) suggest that researchers should develop meaningful and clearly defined categories for coding. Gero (2010) has suggested that the Function-Behaviour-Structure (FBS) ontology provides a common framework for representing design knowledge and allows consistency in coding verbal protocols. However, while this may offer consistent and clearly defined categories, the meaningfulness of the data is still dependent on the research objective. For example, the FBS ontology is poorly suited to provide insights into the use of cognitive heuristics in design.

The current research is primarily concerned with biased cognition during concept generation and evaluation, which is not necessarily well examined using any existing methodologies. Instead the approach taken here, and by Cheong et al. (2012) and Hallihan et al.

(2012), combined structured coding and qualitative observation to gain new insights into the cognitive processes of interest. This approach, of studying creative phenomena from the bottom up, has been supported as a legitimate means to develop new insights in design research

(Brown, 2010). The following section presents research that relied on the qualitative analysis of verbal protocols collected from participants engaged in a realistic design task. 49 4.2. Study 1: Method and Procedure

4.2.1. Participants. Thirty engineering students (28 males and 2 females) from a fourth-year mechanical design course at the University of Toronto volunteered to participate in the study. Participation was part of a voluntary design-by-analogy lab exercise during the course. All data collected came from students who consented to have their design session audio- recorded and to have the data used for research purposes. Participation lasted approximately 60 minutes; participants were not monetarily compensated, instead they were remunerated with the design experience gained during the exercise. The University of Toronto’s Social Sciences,

Humanities and Education Research Ethics Board granted approval for this research.

4.2.2. Procedure. The lab exercise required participants to generate solutions for an engineering design problem using a biological analogy as a source of inspiration. Three design problems were provided, with each problem having a corresponding description of a biological phenomenon as the source of analogy (see Appendix I). Three to four students were assigned to a group and each group worked on a single design problem. There were three lab stations with three groups at each station (see Table 2.2). While one group worked on the design problem, the two observing groups at the lab station were instructed to categorize the designing group’s activities (e.g., design fixation, correct/incorrect analogical transfer).

TABLE 2.2. Participant groups and assigned design problems. Lab Station Design Group # of Students Design Problem 1 4 Promotional Mailing A 2 3 Authorized Disassembly 3 3 Wet Scrubber 4 3 Wet Scrubber B 5 4 Promotional Mailing 6 3 Authorized disassembly 7 3 Authorized Disassembly C 8 4 Wet Scrubber 9 3 Promotional Mailing

Each group was given 20 minutes to generate solutions for the design problem. At the beginning of each 20-minute session, each member of the design group was provided with a

50 written copy of the design problem and relevant biological phenomenon. One group (Group 9) used only 12 minutes and stated they could not generate any more concepts.

The order of problems was counterbalanced (using a Latin Square) between each lab station to control for problem effects. However, it is reasonable to expect the presence of a learning effect for the second and third design groups at each station, since they had the benefit of observing the preceding groups.

Design session mediators. A research assistant was assigned to each lab station to facilitate and audio-record the design sessions. To control for any confounding effects introduced by the research assistants, they were provided with a script to handle potential questions from students, and were instructed not to contribute to the design process. The research assistants were only to intercede when design progress slowed or the students had settled on a design solution. After 20 minutes, the research assistant stopped the design session and provided the next group with the corresponding design problem.

Design protocols. Participants in each design group were instructed to discuss their ideas during the design process; these verbalizations were audio-recorded and transcribed for analysis, however participants were not asked to verbalize all of their thoughts.

Two researchers transcribed the audio recordings from each design group. After each transcript was generated, it was cross-reviewed by the other researcher to verify its accuracy.

Some audio data was not interpretable (e.g., multiple designers speaking at once, designers murmuring very quietly, etc.) and this data was excluded from further analysis. These transcripts constitute the data source used for the qualitative analysis; an example transcript can be seen in

Appendix J.

4.2.3. Qualitative coding. As previously mentioned, the coding scheme used for qualitative analysis should be structured to inform, in an unbiased fashion, the research question of concern (Merriam, 2009). A central component of confirmation bias is that it manifests itself 51 in a tendency to seek out or interpret evidence in a way that will confirm pre-existing beliefs.

Therefore, this analysis was structured to first identify designer beliefs, and then evaluate instances of designers seeking or interpreting evidence pertaining to those beliefs as either confirmatory or disconfirmatory (an example of a coded transcript can be seen in Appendix J).

Coding beliefs. The protocols were segmented based on individuals’ statements, as recommended by Ericsson and Simon (1993). A belief was coded in a segment as any instance when a participant verbalized a statement that conveyed his or her intent to influence the design process in a desired direction (e.g., suggesting a design strategy or providing feedback regarding the current design solution). Knowledge of the participant’s intent within the context of the design process established the nature of the belief. Given that beliefs are subject to change, instances when a participant stated conflicting beliefs were identified, and subsequent coding of confirmation and disconfirmation focused on only the most recently affirmed belief.

Coding confirmation and disconfirmation. Once belief segments were identified, subsequent segments were identified in which a participant either sought or interpreted information that was relevant to the previously identified belief. That instance was then coded as either confirmatory (an attempt to validate or support the belief) or disconfirmatory (an attempt to invalidate or criticize the belief); ambiguous cases were excluded to mitigate bias. While

Ericsson and Simon (1993) generally recommend that a segment be coded based on information contained within the segment itself, they acknowledge that segments can be coded based on information from preceding or subsequent segments to offer additional context and resolve ambiguity. In this case, coding a segment as confirmatory or disconfirmatory is clearly dependent on information from previous segments that contain beliefs.

An example of a confirmatory case is seen in Figure 2.1; the biological phenomenon was an Emperor Penguin’s thermoregulatory capability, the problem asked participants to improve the efficiency of a wet scrubber that removes pollutants from exhaust gases (see Appendix I). 52 Designer A makes a statement coded as a belief that A: I’m thinking that the penguin’s feet really looks the shape of the penguin’s feet (part of the analogy) like the scrubber, I’m not really sure of the shape of should be incorporated into the design solution. the scrubber, but I, I, [sic] I think the scrubber looks like the feet of a penguin. For the next several minutes the group discusses C: I don’t think the penguin’s feet is uh important, potentially relevant features of the analogue. Designer like in this example. It’s actually not relevant, like A repeatedly mentions the importance of penguins’ relevant is the vein and the, and the [sic] artery, how feet and is criticized by Designer C. they create the heat transfer… Designer A temporarily stops discussing this aspect of A: Yea, well we can also bring outside knowledge to the analogy. However, a moment later he makes a this, to this design problem. Um, I think the wet statement that is evidence he is reinterpreting the scrubber looks exactly like a penguin’s feet. I’ve, I’ve problem scenario to confirm his belief. [sic] seen one of them in the, (interrupted) Designer B interrupts and questions the belief. B: You’ve seen one of them? Well, well [sic] what do they look like? Designer A validates his belief. A: They look like a penguin’s feet. FIGURE 2.1. Protocol analysis of a confirmatory case.

There were instances when group members would disagree with each other and present evidence aimed at disconfirming someone else’s belief (e.g., Designer C in Figure 2.1).

However, these cases were not coded as disconfirming because the designer presenting disconfirming evidence could be doing so to support his or her own beliefs. Instead, the evaluation focuses on how the designer being presented with the conflicting information reacts in terms of evaluating the new evidence (e.g., accepting disconfirming evidence is failing to exercise a confirmatory bias and coded as disconfirmation).

Analytical validity. It is possible that participants internally vet their ideas before vocalizing them. This was not a true talk-aloud experiment and participants were working in groups, which may have resulted in pressure on individuals to only vocalize ideas they felt confident about. In addition, participants may have felt pressure to not externalize disconfirmations to avoid appearing critical. Together, these factors could have biased the dialogue towards confirmation. However, Ericsson and Simon (1993) state that under high cognitive load, participants often stop verbalizing during think-aloud protocols. Considering that the design task is presumably cognitively demanding, as it involved analogical comparisons between novel biological entities and mechanical design problems, it is likely participants would have had difficulty verbalizing their thought processes aloud regardless. In addition, the scenario

53 developed represents a realistic design situation. Therefore the results observed are believed to offer an adequate representation of the influence of confirmation bias during concept generation.

4.3. Results

The following sections reports on the results of the protocol analysis, including descriptive statistics and a discussion of insights gained through observation.

4.3.1. Ratio of confirmation to disconfirmation. A total of 107 instances were identified as confirmatory or disconfirmatory. Table 2.3 shows the number of confirmatory and disconfirmatory cases for each group. Figure 2.2 compares the ratio of confirmation to disconfirmation for each group.

TABLE 2.3. Number of confirmatory and disconfirmatory cases per group. Group Number 1 2 3 4 5 6 7 8 9 Total Confirming 11 13 8 10 6 3 16 13 4 84 Disconfirming 3 5 1 3 0 0 5 6 0 23 Total 14 18 9 13 6 3 21 19 4 107

1.0 0.9 0.8 0.7

0.6 Disconﬁrm 0.5 Conﬁrm 0.4 0.3 0.2

Percent of Cases (%/100) 0.1 0.0 1 2 3 4 5 6 7 8 9

Group Number FIGURE 2.2. Ratio of instances of confirmation to disconfirmation by group.

A chi-square goodness-of-fit was used to determine whether cases of confirmation and disconfirmation were equally likely, based on the total number of observations with the expected value for each cell set at chance assuming no bias existed (53.5). The results indicated that confirmation was significantly more likely than chance, and disconfirmation was

54 significantly less likely than chance X2 (1, N = 107) = 33.64, p < .001. The average ratio across all groups was 83% confirmation and 17% disconfirmation (SD = 12.1%). The data indicate that participants’ discussions were biased towards confirmation during the lab exercise.

4.3.2. Qualitative observations. While qualitatively coding the protocols, observations were made that offer additional insight into the nature of confirmation bias in concept generation. While these observations cannot be statistically validated, they may be of interest to practicing designers as well as design researchers.

Discounting factual evidence. When designers hold beliefs that can be contradicted by factual evidence, it is reasonable to assume they will fail to demonstrate a confirmatory bias.

However, instances of belief perseverance in the face of contradicting evidence were observed, which often led participants to misinterpret or ignore relevant information. For example, in one instance a participant thought collecting demographic data would be a useful strategy to solve the promotional mailing problem (see Appendix I). A group member stated that the design brief specified demographic data was not available (which was correct). Still the former participant attempted to persuade the group that gathering demographic information was a useful strategy, a non-optimal response given the feedback he had received. In this way, confirmation bias could contribute to design fixation or an unwillingness to compromise on design ideas; if designers discount or ignore the criticisms of others in order to maintain a belief, they will be unable to appropriately consider the value of alternatives.

Confirming analogies. Participants were given a design task that required them to use a pre-determined biological analogy to inspire solutions for a specific design problem. When participants developed solutions that utilized some aspect of the analogy, they frequently failed to consider if the analogy was being applied inappropriately. This tendency may have contributed to improper analogical transfer. The ability to identify relevant differences between a target design and source analogue may facilitate analogical reasoning. 55 Seeking validation. Confirmation bias may influence the way designers question each other. Participants frequently asked affirming questions (e.g., “What part of this idea do you like?”). However, validating the strengths of existing concepts does little to better the design situation, since these questions do not solicit information that would be informative to improve concept quality. If designers sought information that highlighted flaws in their ideas they would be better equipped to resolve those issues, resulting in improved concepts.

Confidence. An individual’s perceived confidence regarding their knowledge of the design problem or biological phenomenon was believed to influence their reliance on the confirmation bias. Participants who were seen as highly confident seemed more resistant to disconfirming evidence. However, relying on confirmation bias during decision-making has been demonstrated to inflate confidence (Nickerson, 1998). Given the observational nature of this analysis, determining the direction of this relationship is not possible. In addition, the assessment of confidence was determined by discussing this trait with the research assistants who observed the design sessions, and is relatively subjective. However, the established relationship between confidence and confirmation bias suggests that these factors would likely interact and have the potential to influence designers.

Disconfirmation. Instances of disconfirmation were observed when participants accepted evidence that contradicted a design belief, or when they identified limitations of their own ideas. In both of these cases disconfirmation was associated with a perceived lack of confidence in the belief in question. Designers who lack confidence in their ideas might be quick to self-criticize, or be less likely to defend their ideas.

Design criticism. Participants were often hesitant when criticising the ideas of other group members, which could arise from multiple causes (e.g., lack of self-confidence, courtesy, etc.). However, even when criticisms were expressed, they were often vague or irresolute. This unfortunately makes it easier for a designer to dismiss criticisms, and perpetuates a confirmatory 56 bias. Designers were regularly observed failing to see a flaw in their design strategy until they were criticized multiple times with the flaw explicitly pointed out (i.e., specific criticism was more effective than general criticism).

A possible antecedent of this behaviour is perpetuated through the principles of brainstorming. A central principle of brainstorming is that criticism is not allowed (Dieter,

2000). However, criticism facilitates the identification of opportunities for improvement, and is an integral part of knowledge construction in design (Bardzell et al., 2010). Encouraging designers to withhold criticism could be fostering a culture that is ineffective in offering valuable criticism (i.e., providing sufficiently detailed criticism, communicating criticism effectively) and one that is unable to respond to criticism appropriately (i.e., recognizing the value of criticism, maintaining a sense of self-efficacy in the face of criticism). While deferring judgment in brainstorming may encourage divergent thinking, designers will also benefit if they recognize the value of criticism and effectively offer and respond to it. In addition, the value of criticism is inherently tied to the design process being used. Design-by-analogy is a unique situation because concepts that incorrectly apply the source analogue are easily identified, however the absence of this distinction in brainstorming may limit criticism’s value.

4.4. Protocol Analysis Limitations

It is necessary to acknowledge that because the researcher coded the verbal protocols, instead of relying on independent raters, the validity and reliability of the reported analysis is subject to researcher bias. However, even when using independent raters to code design protocols, the raters must be trained to the point that they can accurately identify the phenomena of interest. Part of the reason being that the phenomena of interest are abstract concepts and interpreting them within the context of a design session is a difficult task. Regardless, the present protocol analysis could be improved upon by having independent but trained raters perform the coding in order to improve the reliability of the reported observations. Instructional 57 material to facilitate this was prepared (see Appendix K) but not implemented. Having acknowledged this, the coding structure was developed and adhered to so as to attempt to mitigate researcher bias, and the results of the analysis are robust in indicating that confirmation bias is present during concept generation.

5. STUDY 2: MITIGATING CONFIRMATION BIAS IN CONCEPT EVALUATION

The results of Study 1 suggest that confirmation bias is present among novice designers engaged in a biomimetic-design concept generation exercise. The purpose of the second study was to examine the effectiveness of decision matrices as tools to mitigate confirmation bias during concept evaluation. This study was motivated by previous research demonstrating that formalized decision matrices can improve forecasting accuracy in information analysts by reducing cognitive biases (Brasfield, 2009).

The study is divided and presented based on two problems participants were given.

Problem 1 was intended to replicate an earlier experiment on confirmation bias (Wason, 1968) in order to determine if the participant sample in this study showed a similar propensity towards confirmation as samples in previous experiments. Problem 2 was intended to examine the value of using decision matrices to mitigate confirmation bias during concept evaluation.

5.1. Participants

Sixteen participants (2 female, 14 male) from the University of Toronto participated in the study (completing both problems). Participants were recruited from a University of Toronto graduate residence, as well as from the department of Mechanical and Industrial Engineering.

Participation lasted approximately 45 minutes and participants were compensated $10 for their time. The University of Toronto’s Social Sciences, Humanities and Education Research Ethics

Board granted approval for this research.

58 5.2. Problem 1

Problem 1 was based on Wason’s (1968) card task, in which participants were asked to test the condition: If a card has a vowel on one side, it has an even number on the other side.

Participants are shown 4 cards with: a vowel, a consonant, an even number, or an odd number on the side facing up (see Figure 2.3), and asked to select the cards they think are necessary to test the condition. This task simplifies to a test of the condition If P (vowel) then Q (even number) by selecting among four alternatives that represent: P, NOT P, Q, and NOT Q. The only choice that exclusively allows participants to falsify (disconfirm) the rule is NOT Q, as selecting

P could be motivated by either a positive or negative test strategy.

E K 4 7

P NOT P Q NOT Q

FIGURE 2.3. Alternatives in Wason’s (1968) confirmation bias experiment.

Wason (1968) observed that all participants selected P as necessary to test the condition, and approximately 77% of participants selected Q. Selecting Q only allows one to confirm If P then Q, as observing If NOT P then Q does not invalidate the condition. Very few participants

(28%) selected NOT Q and fewer still (17%) selected NOT P. This is evidence of confirmation bias in the evaluation of a condition that individuals have no vested interest in and demonstrates a tendency to attempt to determine if a belief is true, rather than if it is false.

5.2.1. Procedure. Wason’s original task was modified to provide participants with a test condition more relevant to design. Participants were given Problem 1 (see Appendix L), which instructed them to evaluate the belief: Washing machines that are highly water efficient are also highly energy efficient. Participants were shown a set of stimuli (see Figure 2.4) representing the conditions P (Water Efficient), NOT P (Water Inefficient), Q (Energy

Efficient), and NOT Q (Energy Inefficient). They were then instructed to pick two of the four

59 machines that they would want to learn the remaining information about (the relative energy or water efficiency) in order to optimally evaluate the belief. Participants exhibiting a confirmatory bias were expected to test the belief by examining the P and Q conditions. Participants must select the NOT Q to demonstrate an exclusively negative (disconfirmatory) test strategy.

Participants were instructed to record their selections on a blank sheet of paper, in addition to listing any relevant considerations that contributed to their decision. Participants were given as much time as they felt was necessary to make their decision. After they had made a decision, the researcher asked them to verbally relate their decision-making process to clarify any ambiguity in the written reports and ensure they were properly interpreted.

P NOT P Q NOT Q FIGURE 2.4. Problem 1 alternatives; lettered conditions (P, NOT P, etc.) not shown.

5.2.2. Results. The data collected from participant responses to Problem 1 can be seen in Table 2.4. All participants selected the Water Efficient (P) alternative to evaluate the belief, however the remaining decisions were distributed between the other alternatives. One participant decided to select only the Water Efficient machine, and none of the others, resulting in a total of 31 selections from 16 participants. The frequencies observed are reported, however formal statistical testing is not performed as a cursory examination indicates participants were no more or less likely to select NOT P, Q, or NOT Q.

TABLE 2.4. Participant responses for Problem 1. Water Efficient Water Inefficient Energy Efficient Energy Inefficient

(P) (NOT P) (Q) (NOT Q) Participant 16 (100) 6 (37.5) 5 (31.3) 4 (25.0) Selections (%)

60 Only 25% of participants selected the NOT Q case in their decision. The remaining 75%

(including the participant who selected only the High Water Efficiency machine) chose instances that would allow them to confirm the belief, or were irrelevant to evaluating the belief.

5.2.3. Discussion. The results presented in Table 2.4 show that 4 of 16 participants selected the combination of P and NOT Q, in the experiment of interest from Wason (1968), this ratio was 4 of 18. However, Wason allowed participants to select more than two alternatives, and selections of three or four alternatives accounted for 1/3 of all participant selections.

Therefore, while the frequency of selections evidencing disconfirmation is relatively similar to the previous study, the observed effects are in slightly different contexts (i.e., participants in

Wason’s experiment had more combinations of alternatives to choose from).

The differences here are not statistically analyzed, as the observed frequencies suggest that no statistical difference exists between the NOT P, Q, and NOT Q, alternatives. However, the primary purpose of this problem was to determine if participants in the present sample behaved similarly to participants in previous ones, and based on a comparison of the observed selection of the P and NOT Q combination this seems to be the case, although participant selections for the NOT P and Q alternatives were quite different, possibly a result of limiting participants’ selection to 2 of the 4 alternatives.

Pragmatism. Participant notes indicated that of the 12 individuals who failed to select the energy inefficient machine to disconfirm the belief, 11 explicitly stated pragmatic considerations as a factor (e.g., establishing a causal relationship between water and energy efficiency, gathering evidence to identify correlation). Pragmatism has been forwarded as a possible reason for individuals’ preference for confirmatory test strategies. Friedrich (1993) gives multiple reasons for the value of pragmatism in decision-making (over detecting objective truths) including the maladaptive nature of adopting negative test strategies. Friedrich suggests that individuals demonstrate a confirmatory bias because it allows for better decision-making in 61 real life. For example, if an individual has a belief that a certain berry is poisonous it would seem unnecessarily reckless to adopt a negative test strategy to prove the belief wrong. The participants’ written reports support the pragmatic explanation, and although the majority fail to consider disconfirming evidence, their decisions are not specifically maladaptive in this context.

Education effects. Interestingly, three of the four participants who selected the disconfirmatory case were law students. Post-experiment interviews revealed that these participants recognized that a statement in the form If P then Q does not imply If Q then P. This eliminates the Q alternative as an option. These participants also all reported that they selected the NOT Q condition because they understood it could provide evidence to disprove the statement. Cosmides (1989) hypothesized that individuals are better at detecting disconfirming evidence when it can be perceived as the violation of a social contract. The participants’ legal education may help them to perceive contractual violations in a wider range of tasks than other participants. However, this effect did not seem to influence performance in Problem 2.

5.3. Problem 2

Problem 2 (see Appendix M) was developed to provide insights into the effectiveness of decision matrices in mitigating confirmation bias during a concept evaluation task. The instructions given to participants for using a decision matrix were adapted from the Analysis of

Competing Hypotheses (ACH) methodology. ACH was developed by Heuer (1999) as a decision-making tool to improve the forecasting accuracy of information analysts. The 8-step method helps analysts generate a matrix that facilitates the comparison of alternative hypotheses and the evaluation of the relevance and diagnostic value of gathered evidence. It has been demonstrated to reduce reliance on cognitive biases, including confirmation bias, in complex decision-making tasks with uncertain outcomes. In addition, it was demonstrated that ACH aids participants in evaluating more information regarding a decision than participants relying solely on intuition (Brasfield, 2009). 62 The Modified Analysis of Competing Hypotheses (MACH) procedure was developed for participants to use in Problem 2. The modified version was reduced to 5 steps and instructed participants to generate a matrix to compare and evaluate conclusions regarding the decision task with respect to the available evidence (see Appendix N.1).

5.3.1. Procedure. Participants were first provided with a brief background on design fixation (Jansson & Smith, 1991) and directed to evaluate the validity of the belief: The presence of an example causes designers to fixate and incorporate elements of the example in their solutions. To evaluate this belief they were shown the example solution for the design problem discussed in Chapter 1, along with six concepts developed to solve the problem (see

Appendix M and Figure 2.5). Participants were told these six concepts had been generated by individuals who had been exposed to the example solution. Two of the concepts (1 & 3) incorporated multiple elements of the example solution, while the others (2,4,5,6) did not. These design concepts were selected because they provide the participants with substantial evidence to disconfirm the belief, and an ideal evaluation should reflect this ratio.

Example

1 2 3

4 5 6 FIGURE 2.5. Problem 2 evaluation concepts 1-6 (Hallihan & Shu, 2011) and example concept (Perttula & Liikkanen, 2006). Concept descriptions can be seen in Appendix M.

63 Participants in the control group were provided with instructions to evaluate the concepts intuitively (see Appendix N.2) and to record relevant considerations in evaluating the belief as point form notes on a blank sheet of paper. Participants in the treatment group were provided with instructions to evaluate the belief using the MACH matrix (see Appendix N.1).

Before beginning to solve each problem, participants were questioned to ensure they understood the problem as intended by the researcher. After completing each problem, participants were interviewed, which provided an opportunity for the researcher to ensure they were properly interpreting the participants’ written notes.

Participants were told that the average completion time for this task was 15 minutes, but that they would have as much time as they wanted to reach an optimal conclusion. Their performance was timed to allow for comparison of the duration of problem solving between the treatment and control conditions. Timing began once participants read and indicated they understood the instructional materials and began problem solving.

5.3.2. Coding confirmation and disconfirmation. Participants in the control group were instructed to use blank sheets of paper and point form notes to record any relevant information that they considered during their evaluation. Participants in the treatment condition externalized their evaluation using the MACH matrix. These self-generated records were analyzed to measure confirmation and disconfirmation. Written documentation indicating the consideration of evidence, or argument for, confirming the fixation hypothesis was counted as one instance of confirmatory evidence. Similar documentation that disconfirmed the fixation hypothesis was counted as one instance of disconfirmatory evidence. The total number of instances were counted for each participant. Examples of coded data can be seen in Figure 2.6

(matrix) and Figure 2.7 (no matrix); the original sheets can be seen in Appendices O.1 and O.2.

64 Degree of Features of Design from Outside Features of Design From Example Fixation Sources - Overhead release of water(C) - Fed by water line(C) Concept 1 - Sprinkler head(C) - Ball float valve(D) High Fixation - Periodic release at intervals (requiring timer)(C) - Valve of some kind(C) - Water wheel release(D) Concept 2 - Fed by water line(C) - Continual release of water at fixed Medium Fixation - Overhead release of water(C) tempo (no timer required)(D) - Overhead release of water(C) - Sprinkler head(C) Concept 3 - Natural cloud source /fed by - Fed by water line(C) High Fixation rainwater(D) - Periodic release at timed intervals (requiring time)(C) - Dripper release(D) - Continual release of water at Concept 4 natural tempo(D) - ? [sic] Low Fixation - Soil fed stream(D) - No water line(D) - No timer required(D) - External movement brings plant to water (instead of bringing water to Concept 5 plant)(D) - Timer required(C) Low Fixation - Hydraulic lift required(D) - No flow of water stream(D) - Higher relative energy required(D) - No water stream(D) - No timer required(D) Concept 6 - ? [sic] - No external movement(D) Low Fixation - Sponge fed(D) - Soil fed hydration(D) Figure 2.6. Coded participant matrix: 12 disconfirming(D) and 18 confirming(C) instances.

Top Left: incorporates water line(C) and a similar looking sprinkler head(C) Top Middle: incorporates a house water line(C) Top Right: incorporates many elements(C), except the water line(D) Bottom Left: seems to incorporate no elements(D) Bottom Middle: incorporates predetermined intervals(C) Bottom Right: seems to incorporate no elements(D) Figure 2.7. Coded participant notes: 5 disconfirming(D) and 3 confirming(C) instances.

5.3.3. Results. The data collected from Problem 2 are seen in Table 2.5. Three participants exhibited behaviour that was believed would unduly influence the analysis.

Participant 7 was assigned to the treatment group, but did not follow the MACH procedure as outlined. Participants 6 and 12 were assigned to the control condition, however they utilized matrices to formalize their decision process in a way that simulated the treatment condition.

While it was originally intended to test the hypothesis that the MACH manipulation would

65 mitigate confirmation bias, these three cases confounded the original comparison. Therefore, the comparison focused on participants who utilized matrices to formalize their decision process with participants who relied on intuition without a matrix (Matrix: Yes, No).

TABLE 2.5. Conditions and data for problem 2. Sub Group Matrix Major Confirm Disconfirm Time (min) No. 1 MACH Yes Zoology 8 10 15.7 2 Control No Genetics 4 4 6.3 3 MACH Yes Sociology 7 8 14.4 4 Control No Medicine 3 1 6.8 5 MACH Yes Law 9 9 20.2 6 Control Yes Law 12 18 20.5 7 MACH No Law 1 2 9.7 8 MACH Yes Law 4 1 10.2 9 MACH Yes Eng. 11 16 35.9 10 Control No Eng. 8 6 19.3 11 MACH Yes Eng. 5 11 17.9 12 Control Yes Eng. 6 18 22.8 13 Control No Eng. 5 3 11.8 14 Control No English 7 11 11.6 15 Control No Law 4 2 5.3 16 Control No Eng. 2 3 15.2

Effect of matrix. A one-way multivariate analysis of variance (MANOVA) was used to examine the differences between groups (Matrix: Yes, No) with respect to evidence evaluated

(confirming, disconfirming). There was a statistically significant difference between groups,

F(2,13) = 4.95, p = 0.025, Wilks’ λ = 0.57, partial ε2= 0.43.

Follow-up comparisons (see Figure 2.8) were performed using independent samples t- tests, with the Bonferroni Correction (α/2 = 0.025). There was a statistically significant difference, t(14) = 2.69, p = 0.018, in the amount of confirming evidence evaluated: Matrix (M

= 7.75, SE = 1.00), No Matrix (M = 4.25, SE = 0.84). There was also a statistically significant difference, t(14) = 3.15, p = 0.07, in the amount of disconfirming evidence evaluated: Matrix (M

= 11.38, SE = 2.05) No Matrix (M = 4.00, SE = 1.13).

66 14 Conﬁrming 12 Disconﬁrming

2 Evidence Evaluated (Instances) 0 Matrix No Matrix

Figure 2.8. Confirming and disconfirming instances evaluated between matrix and no matrix conditions.

Effect of time. There was a strong and statistically significant correlation between the amount of time participants spent solving the problem and the quantity of evidence evaluated: confirmatory (r = 0.72, p < 0.01), disconfirmatory (r = 0.76, p < 0.01). There was no interaction between time and the type of evidence evaluated. Although the Matrix group identified significantly more evidence than the No Matrix group, there was no statistically significant difference in the number of items evaluated per minute.

Participant conclusions. With respect to the belief participants were asked to evaluate, twelve of the sixteen concluded that there was evidence available to both support and reject the fixation hypothesis. The remaining four participants (4, 8, 10, & 16) concluded that the fixation hypothesis was well supported by this data. Participants 4 and 8 evaluated more confirmatory evidence than disconfirmatory, and participants 10 and 16 evaluated more disconfirmatory evidence than confirmatory. The latter case may be a result of those participants failing to properly evaluate the diagnostic value of contradictory evidence, a known consequence of confirmation bias (Nickerson, 1998).

5.3.4. Discussion. The primary purpose of this analysis was to evaluate the effectiveness of decision matrices as tools to mitigate confirmation bias in concept evaluation. A

67 number of methods already exist in design textbooks that utilize matrices to facilitate concept evaluation (e.g., Pugh’s Concept Selection Method, Weighted Decision Matrices, Analytic

Hierarchy Process [Dieter, 2000]). The current findings indicate the use of formalized matrices resulted in participants identifying more disconfirmatory cases than participants that did not use matrices. Because the concepts evaluated (see Figure 2.5) provided more evidence against the fixation belief than for, and after comparing the ratio of confirmatory to disconfirmatory evidence evaluated between groups (see Figure 2.8), it can be concluded that the use of matrices allowed participants to perform a less biased or more thorough evaluation.

The effect of time. It is possible that the observed effects were due to the increased time spent on the task in the Matrix condition relative to the No Matrix condition. However, participants in both cases decided themselves when they “reached an optimal conclusion.”

Therefore, the observed differences in the time spent on the task could have also been facilitated through the use of a matrix. Future experiments could control for this variable to determine whether time itself could produce the observed effects. However, while it would be relatively easy to require all participants spend the same amount of time evaluating the concepts, the method used here is more realistic as participants decided when they had reached an adequate conclusion. In addition, based on observations of participants generating concepts in the experiment described in Chapter 1, if too much time is provided participants may stop working on the problem once they are content with their conclusions, regardless of time remaining.

Resistance to formalized methods. Three of the participants reported that using the

MACH matrix was an unnatural way for them to think, including Participant 7 who actually seemed incapable of using the MACH table as an evaluation tool. Previous research has also indicated that individuals often resist the use of formal decision-making methods, instead preferring to rely on intuitive methods (Brasfield, 2009). This resistance limits the utility of any formalized method and may negate its potential benefits if participants apply it incorrectly. 68 Cognitive effort. The resistance to use a procedure, and the failure to properly apply it, may arise if the procedure requires increased cognitive effort to utilize. Given a limited information-processing capacity, any method that requires additional processing may result in decreased cognitive effort allocated to other concurrent tasks. However, even among novice matrix users, the written record of the decision process could lessen working-memory load, freeing up cognitive resources. Future research could examine the cognitive demand imposed on participants using these methods through measures that assess cognitive workload (e.g., NASA

TLX [Hart & Staveland, 1988]). Anecdotally, the two participants in the control condition who used matrices spontaneously evaluated the most evidence out of all the participants, and were both above average in the number of items evaluated per minute. This suggests that individuals who can use matrices intuitively have an advantage over those who cannot. Educating individuals on the use and benefit of these methods would likely increase their utility.

Design relevance. The use of matrices to formalize the process of concept evaluation is not new to design. However, this research highlights another benefit of their use, namely the mitigation of cognitive biases, specifically confirmation bias. Comparing the effectiveness of different existing concept evaluation methodologies in mitigating cognitive bias is an interesting area for future research. One of the benefits of matrices in mitigating confirmation bias is that they allow individuals to see how arguments that support the selection of one concept may apply equally well to an alternate concept. A benefit of the ACH procedure specifically is that it encourages users to generate disconfirming evidence; this formalizes the process of criticising ideas and may make it easier to both administer and respond to criticism.

Additional insights from the protocol analysis suggest that successful methods should compensate for individuals’ avoidance of criticism, preferential treatment of initially generated concepts, false sense of confidence, and failure to consider disconfirming evidence.

69 Empirical limitations. Although statistically significant differences were observed between experimental groups, the relatively small sample size limits the statistical reliability of these findings. In addition, the sample included both non-engineering students and engineering students. While it was not the initial intent to sample non-engineering students, their inclusion in the study provided valuable insights (e.g., the possible moderating effect of a legal education).

However, as mentioned, (and demonstrated in the findings presented) in Chapter 1, engineering students likely view design problems differently than non-engineering students.

Several participants (6, 12, 7) were included in groups for analysis that they were not originally assigned to. One option would be to leave these individuals out of the analysis completely; if this is done the same trends are observed, however the probability of these effects being due to chance approaches p = 0.07. Considering the Bonferroni correction was applied, this trend is far from statistically significant, however the direction of the relationship remains unchanged and the decrease in sample size is somewhat accounted for by a decrease in the sample variance once these participants are removed.

6. SUMMARY AND CONCLUSIONS

It was hypothesized that confirmation bias was an influential bias in design cognition.

The presence of confirmation bias during concept generation was identified through the analysis of design protocols collected from engineering students engaged in a biomimetic design practical session. Results of this analysis revealed that confirmation bias could contribute to design fixation, a failure to identify the misapplication of analogies, and a tendency to ignore relevant contradictory information. In addition, the influence of confirmation bias may be magnified by overconfidence and a hesitance to be critical of others. Finding ways to encourage designers to voice criticisms clearly and handle criticism effectively may mitigate confirmation bias during concept generation. Negative feedback and increased personal accountability for 70 decisions have both been shown to decrease overconfidence (Arkes et al., 1987). Interestingly, avoiding criticism reduces the presence of both of these factors, and overconfidence is hypothesized to contribute to an increased reliance on confirmation bias.

The results of the second study offer insight on the use of decision matrices as concept evaluation tools. Although these methods can mitigate confirmation bias, they can be met with resistance and misapplied, limiting their benefit. In addition, the cognitive effort required to use a method could hypothetically lead to an increased reliance on cognitive heuristics to offset this demand. Adequate training on the use of any new methodology will likely address these issues.

Jin et al. (2006) proposed that formalized methods could enhance the concept generation process. The focus of this study was on how formalized methods may be used to mitigate the influence of cognitive heuristics and biases in concept generation, however these tools may have other benefits. One possibility is that the use of tools like ACH can help individuals determine the depth and breadth (quality and quantity) of the concept generation process and adjust accordingly. A matrix that over emphasizes alternative concepts without incorporating relevant evidence or supporting arguments for those concepts, may indicate that the concept generation process is overly focused on quantity and not quality. On the other hand, too few or highly similar concept alternatives may indicate fixation.

Finally, while the majority of the research in this chapter is concerned with confirmation bias, the application of literature regarding all cognitive heuristics and biases is a promising area for future research. This chapter outlines a number of these that are thought to be particularly relevant to design.

71 CHAPTER 3 THESIS SUMMARY

The goal of this research was to gain a better understanding of design cognition through the application of psychological theories. While the theories discussed (i.e., cognitive heuristics and long-term potentiation) are distinct and may seem unrelated, they both afford a means to understand designer biases in concept generation and evaluation. It is through this understanding that new and more effective methods to mitigate biased design cognition can be developed.

Long-Term Potentiation and Design Creativity

In Chapter 1, an argument is forwarded that emphasizes long-term potentiation as a phenomenon directly relevant to design cognition. This argument in itself is not particularly novel, as LTP must influence cognition if it changes the way neurons communicate information.

What is novel is that this argument is expanded to discuss creativity specifically, which has yet to be publically reported. Then, in an attempt to demonstrate the actual value of considering

LTP in design cognition, a theoretical explanation of design fixation arising from LTP is presented. The principal value of this theoretical discussion for the design community is to provide a new perspective that can further the understanding of biased design cognition. In addition, hypothetical interventions for mitigating fixation through enhancing awareness, based on detection using metrics of memory, are presented. While these are not empirically validated, the link between fixation and memory is supported by previous research.

The empirical study conducted did not validate a hypothesis based on one application of this theory (i.e., the influence of physical activity on the efficacy of a defixation exercise).

However, the results still offer valuable insights to contribute to the research community’s

72 understanding of design methodology and design fixation. It was demonstrated that individuals with engineering backgrounds behave differently from participants with non-engineering backgrounds; although not surprising or novel in itself, the result emphasizes how educational differences may contribute to design fixation. In addition, the finding that engineering students in the physical activity condition generated more solutions without generating higher quality solutions supports the independence of concept quantity from quality. Although the reliability of this finding is limited due to issues associated with running multiple statistical tests. Finally this research demonstrated, unintentionally, how the metrics used to assess fixation dictate the outcomes from fixation experiments. For instance, even when evaluating fixation on an example with a relatively simple coding scheme (i.e., similar or not), raters show surprisingly high levels of disagreement. This is likely because the concepts participants generated were not always congruent with the coding scheme based on functional categories. Metrics used to quantify fixation should be sensitive to variability in participant responses during concept generation exercises. While there are multiple ways to assess fixation, the design literature offers no one definitive and accepted method. Establishing measures that are not influenced by participants’ interpretation of the design task, or concept quantity, are promising areas for future research.

Cognitive Bias and Confirmation in Design

Chapter 2 discusses cognitive heuristics and biases that may be of particular relevance to understanding design cognition, with an emphasis on the role of confirmation bias. The overview of heuristics deemed to be the most relevant to design is largely meant to be an introduction to designers unfamiliar with the literature. The primary focus of this chapter was to illuminate the influence of confirmation bias during concept generation and concept evaluation.

The first study presented analyzed verbal protocols to determine whether or not confirmation bias was present among designers during a biomimetic concept generation 73 exercise. The results provide evidence for the presence of confirmation bias, however the reliability of this conclusion cannot be discussed from the perspective of reliability coefficients between independent raters. While this is not optimal, previous researchers have performed qualitative analyses in a somewhat similar fashion. Chrysikou and Weisberg (2005) used non- independent raters to analyze written protocols in order to identify fixation, afterwards recruiting an independent rater to code 30% of a single protocol in order to provide a measure or reliability. Christensen and Schunn (2007) analyzed verbal protocols from an analogical-design session in a similar fashion, using an independent rater to analyze 18% of the total data in order to provide some measure of reliability. While these cases still partially rely on independent raters, they rely more heavily on the authors performing qualitative analyses themselves. It is this researchers’ belief that qualitative analyses provide more valuable and accurate insights into the phenomena of interest if the raters used have domain expertise, which may require substantial training for independent raters to acquire. In addition, while performing an analysis independently may enhance the reliability of the analysis, arbitration and discussion of discrepancies that arise (methods also previously used in design [Gero & McNeil, 2005]), will enhance its construct validity.

Qualitative observations made during the coding process are discussed, and while they cannot be validated statistically they offer additional insights. Perhaps the most interesting observation relates to the role of criticism during concept generation. It is often espoused that criticism should be avoided during this process to facilitate group creativity, however the observations here suggest criticism may serve the beneficial function of mitigating cognitive bias during design. Reliance on confirmation bias may inflate problems associated with ignoring or avoiding criticism during the conceptual design process.

The second study examined the role of confirmation bias in concept evaluation. The first problem presented was originally intended to compare the participant sample to previous ones 74 with respect to their propensity towards confirmatory test strategies. However, it was observed that educational differences influenced how participants perceived the problem. Law students seemed more likely to view a problem and attempt to disprove a statement than to seek to confirm it, however this educational training did not translate into avoiding confirmation bias in the second problem. Additionally, the majority of participants adopted a pragmatic problem solving approach, which indicated that they were evaluating a belief that was not necessarily maladaptive. However, the results observed in Problem 2 demonstrated that reliance on confirmation bias during concept evaluation could have a detrimental influence.

The second problem was intended to evaluate the effect of using formalized decision matrices on confirmation bias during concept evaluation. Three participants exhibited behaviour that confounded the analysis, and as such the comparison did not specifically evaluate the effectiveness of the MACH procedure developed for this study. However, the results demonstrated that the use of decision matrices mitigated confirmation bias, contributing to a more thorough and less biased concept evaluation. One limitation on this finding is that the time participants spent evaluating the concepts was not controlled for. However, previous research has demonstrated that using formal decision matrices does mitigate confirmation bias (Brasfield,

2009), and in this study it can be argued that the decision matrix at least the very least facilitated the increased time spent during concept evaluation.

Conclusion

While this research is based on well-established psychological literature, its application in the field of design theory and methodology offers both novel and practical contributions.

Theoretical observations on the underlying biology of creative design phenomena await validation, however the argument presented provides a new perspective on creative design phenomena. This research has hopefully convincingly demonstrated the value of considering 75 cognitive heuristics in design cognition, as well as a method to mitigate negative outcomes associated with confirmation bias. Obstacles and design biases that hinder creativity in conceptual design, such as design fixation, must have psychological origins. Understanding the psychology of designers is a necessary step in the development of a complete theory of design.

76 REFERENCES

Adams, R., & Atman, C. (1999). Cognitive processes in iterative design behaviour. Proceedings

of the ASEE/ISEE Frontiers in Education Conference, San Juan, Puerto Rico, 11a6-13 –

11a6-18.

Agans, R., & Shaffer, L. (1994). The Hindsight Bias: The Role of the Availability Heuristic and

Perceived Risk. Basic and Applied Psychology, 15(4), 439-449.

Amabile, T. (1982). Social Psychology of Creativity: A Consensual Assessment Technique.

Journal of Personality and Social Psychology, 43(5), 997-1013.

Amabile, T. (1996). Creativity in Context. Boulder, CO: Westview Press Inc.

Andres-Barquin, P. (2002). Santiago Ramon y Cajal and the Spanish school of neurology. The

Lancet Neurology, 1(7), 445-452.

Arkes, H., & Blumer, C. (1985). The Psychology of Sunk Cost. Organizational behavior and

human decision processes, 35(1), 124-140.

Arkes, H., Christensen, C., Lai, C., & Blumer, C. (1987). Two Methods of Reducing

Overconfidence. Organizational behaviour and human decision processes, 39, 133-144.

Aronson, E., Wilson, T., Akert, R., & Fehr, B. (2006). Social Psychology (3rd ed.), Upper Saddle

River, NJ: Pearson Prentice Hall.

Asch, S. (1946). Forming impressions of personality. Journal of Abnormal Social Psychology,

41, 258-290.

Bardzell, J., Bolter, J., & Löwgren, J. (2010). Interaction Criticism: three readings of an

interaction design, and what they get us. Interactions, 17(2), 32-37.

Barnes, C. (1979). Memory deficits associated with senescence: a neurophysiological and

behavioural study in the rat. Journal of Computational Physiology and Psychology, 93, 74-

104.

Birsch, D., & Fielder, J. (1994). The Ford Pinto Case: a study in applied ethics, business, and 77 society, Albany, NY: State University of New York Press.

Bliss, T., & Gardner-Medwin, A. (1973). Long-lasting potentiation of synaptic transmission in

the dentate area of the unanaesthetized rabbit following stimulation of the perforant path.

Journal of Physiology, 232, 357-374.

Bliss, T., & Lomo, T. (1973). Long-lasting potentiation of synaptic transmission in the dentate

area of the anaesthetized rabbit following stimulation of the perforant path. Journal of

Physiology, 232, 331-356.

Bliss, T., & Collingridge, G. (1993). A synaptic model of memory: long-term potentiation in the

hippocampus. Nature, 361, 31- 39.

Bliss, T., Collingridge, G., & Morris, R. (2003). Introduction: Long-term potentiation and the

structure of tissue. Philosophical Transactions of the Royal Society B; Biological Sciences,

358(1432), 607-611. de Bono, E. (1970). Lateral Thinking. New York, NY: Harper & Row.

Brasfield, A. (2009). Forecasting Accuracy and Cognitive Bias in the Analysis of Competing

Hypotheses. (Master’s Thesis). Mercyhurst College, Erie, PA.

Breedlove, S., Rosenzweig, M., & Watson, N. (2007a). Biological psychology: an introduction

to behavioural, cognitive, and clinical neuroscience (5th ed.), Ch. 3-4, Sunderland, MA:

Sinauer Associates.

Breedlove, S., Rosenzweig, M., & Watson, N. (2007b). Biological psychology: an introduction

to behavioural, cognitive, and clinical neuroscience (5th ed.), Ch. 18, Sunderland, MA:

Sinauer Associates.

Brown, D. (2010). The curse of creativity. Design Computing and Cognition’10, Stuttgart,

Germany: Springer.

Bullock, T., Bennett, M., Johnston, D., Josephson, R., Marder, E., & Fields, D. (2005). The

Neuron Doctrine, Redux. Science, 310(5749), 791-793. 78 Cagan, J. (2007). A look at the emerging science of innovation. Artificial Intelligence for

Engineering Design, Analysis and Manufacturing, 21, 13-14.

Cardoso, C., & Badke-Schaub, P. (2011). Fixation or Inspiration: Creative Problem Solving in

Design. The Journal of Creative Behavior, 45(2), 77-82.

Chapman, L. (1967). Illusory Correlation in Observational Report. Journal of Verbal Learning

and Verbal Behavior, 6, 151-155.

Chapman, L., & Chapman, J. (1969). Illusory correlation as an obstacle to the use of valid

psychodiagnostic signs. Journal of Abnormal Psychology, 74, 271-280.

Chase, W., & Simon, H. (1973). Perception in Chess. Cognitive Psychology, 4, 55-81.

Cheong, H., Hallihan, G., & Shu, L. (2012). Understanding Analogical Reasoning in

Biomimetic Design: An Inductive Approach. Design Computing and Cognition’12, College

Station, TX: Springer.

Chiu, I. & Shu, L. (2010). Potential limitations of verbal protocols in design experiments.

Proceedings of ASME IDETC/CIE (DETC2010-28675), Montreal, QC: ASME.

Christensen, B., & Schunn, C. (2007). The relationships of analogical distance to analogical

function and preinventive structure: The case of engineering design. Memory & Cognition,

35(1), 29-38.

Chrysikou, E., & Weisberg, R. (2005). Following in the Wrong Footsteps: Fixation Effects of

Pictorial Examples in a Design Problem-Solving Task. Journal of Experimental Psychology,

31(5), 1134-1148.

Cline, H. (1998) Topographic maps: Developing roles of synaptic plasticity. Current Biology, 8,

R836-R839.

Collins, A., & Loftus, E. (1975). A Spreading-Activation Theory of Semantic Processing.

Psychological Review, 82(6), 407-428.

79 Cormier, P., Literman, B., & Lewis, K. (2011). Empirically Derived Heuristics to Assist

Designers with Satisfying Consumer Variation in Product Design. Proceedings of ASME

IDETC/CIE (DETC2011-48448), Washington, DC: ASME.

Cosmides, L. (1989). The logic of social exchange: Has natural selection shaped how humans

reason? Studies with the Wason selection task. Cognition, 31, 187-276.

Cotman, C., & Berchtold, N. (2002). Exercise: a behavioural intervention to enhance brain

plasticity. TRENDS in Neurosciences, 25(6), 295-301.

Cross, N. (2001). Design cognition: Results from protocol and other empirical studies of design

activity, In C. Eastman, W. Newstatter & M. McCracken, (Eds.), Design Knowing And

Learning: Cognition In Design Education, pp. 79-103, Oxford, UK: Elsevier.

Derrick, B., & Martinez, J. (1995). Associative LTD at the Hippocampal Mossy Fiber-CA3

Synapse. Social Neuroscience Abstracts, 21, 603.

Dieter, G. (2000). Engineering Design: A Materials and Processing Approach (3rd ed.), New

York, NY: McGraw-Hill.

Dietrich, A. (2004). The cognitive neuroscience of creativity. Psychonomic Bulletin & Review,

11(6), 1011-1026.

Dijksterhuis, A., & Meurs, T. (2006). Where creativity resides: The generative power of

unconscious thought. Consciousness and Cognition, 15, 135-146.

Dunbar, K. (1995). How scientists really reason: Scientific reasoning in real-world laboratories,

In R. Sternberg & J. David, (Eds.), Mechanisms of Insight, pp. 365-395, Cambridge, MA:

MIT Press.

Duncker, K. (1945). On Problem Solving, Psychological Monographs, 58(5), Whole No. 270.

Ericsson, K., & Simon, H. (1993). Protocol Analysis: Verbal Reports as Data, (Revised ed.),

Cambridge, MA: MIT Press.

80 Farmer, J., Zhao, X., Van-Praag, H., Wodtke, K., Gage, F., & Christie, B. (2004). Effects of

voluntary exercise on synaptic plasticity and gene expression in the dentate gyrus of adult

male Sprague-dawley rats in vivo. Neuroscience, 124, 71-79.

Fischhoff, B. (1975). Hindsight ≠ Foresight: The effect of outcome knowledge on judgment

under uncertainty. Journal of Experimental Psychology: Human Perception and

Performance, 3, 349-358.

Fiske, S., & Taylor, S. (1984). Social cognition (1st ed.), Reading, MA: Addison-Wesley.

Fodor, J., & Pylyshyn, Z. (1988). Connectionism and cognitive architecture: A critical analysis.

Cognition, 28, 3-71.

Friedrich, J. (1993). Primary Error Detection and Minimization (PEDMIN) Strategies in Social

Cognition: A Reinterpretation of Confirmation Bias Phenomena. Psychological Review,

100(2), 298-319.

Gabora, L. (2010). Revenge of the “Neurds”: Characterizing Creative Thought in Terms of the

Structure and Dynamics of Memory. Creativity Research Journal, 22(1), 1-13.

Gauthier, I., Tarr, M., Anderson, A., Skudlarski, P., & Gore, J. (1999). Activation of the middle

fusiform ‘face area’ increases with expertise in recognizing novel objects. Nature

Neuroscience, 2(6), 568-573.

German, T., & Barrett, H. (2005). Functional Fixedness in a Technology Sparse Culture.

Psychological Science, 16(1), 1-5.

Gero, J. (1990). Design Prototypes: A Knowledge Representation Schema for Design. AI

Magazine, 11(4), 26-36.

Gero, J., & Kannengiesser, U. (2006). The situated function-behaviour-structure framework.

Design Studies, 25(4), 374-391.

Gero, J., & McNeill, T. (2006). An Approach to the Analysis of Design Protocols. Design

Studies, 19(1), 21-61. 81 Gero, J. (2009, March 26). Developing Principles for a Theory of Design or What Can

Cognitive Science Contribute to a TOD. Meeting of the Design Society Advisory Board,

Cambridge, MA.

Gero, J. (2010). Generalizing design cognition research, In K. Dorst et al. (Eds.),

DTRS8:Interpreting Design Thinking, pp. 187-198, Sydney, AUS: DAB Documents.

Gifford-Jones, W. (1977). What Every Woman Should Know About A Hysterectomy. New York,

NY: Funk & Wagnalls.

Gilovich, T., Griffin, D., & Kahneman, D. (2002). Heuristics and biases: The psychology of

intuitive judgment. New York, NY: Cambridge University Press.

Guindon, R. (1990). Designing the Design Process: exploiting opportunistic thoughts. Human-

Computer Interaction, 5, 305-344.

Guindon, R., & Curtis, B. (1988). Control of Cognitive Processes During Software Design:

What Tools are Needed? Proceedings of CHI’88, Washington, DC.

Haier, R. (1993). Cerebral glucose metabolism and intelligence, In P. Vernon (Ed.), Biological

approaches to the study of human intelligence, pp. 415-426, Norwood, NJ: Ablex.

Hallihan, G., & Shu, L. (2011). Creativity and Long-Term Potentiation: Implications for Design.

Proceedings of ASME IDETC/CIE (DETC2011-48595), Washington, DC: ASME.

Hallihan, G., Cheong, H., & Shu, L. (2012). The Role of Cognitive Bias and Confirmation in

Design Cognition, Proceedings of ASME IDETC/CIE (DETC2012-71258), Chicago, IL:

ASME.

Hart, S., & Staveland, L. (1988). Development of NASA-TLX (Task Load Index): Results of

empirical and theoretical research, In P. Hancock & N. Meshkati (Eds.), Human Mental

Workload, pp. 139-183, Amsterdam, The Netherlands: Elsevier Science.

Hebb, D. (1949). The Organization of Behavior: A Neuropsychological Approach. New York,

NY: John Wiley & Sons. 82 Heilman, K., Nadeau, S., & Beversdorf, D. (2003). Creative Innovation: Possible Brain

Mechanisms. Neurocase, 9(5), 369-379.

Helms, M., & Goel, A. (2012). Analogical Problem Evolution in Biologically Inspired Design.

Design Computing and Cognition’12, College Station, TX: Springer.

Helms, M., Vattam, S., & Goel, A. (2009). Biologically inspired design: Process and products.

Design Studies, 30(5), 606-622.

Heuer, R. (1999). Psychology of Intelligence Analysis, Ch. 8, Washington, DC: Center for the

Study of Intelligence.

Jansson, D., & Smith, S. (1991). Design Fixation. Design Studies, 12(1), 3-11.

Jin, Y., Kim, D., & Danesh, M. (2006). Value Based Design: An Objective Structuring

Approach to Design Concept Generation. Proceedings of ASME IDETC/CIE (DETC2006-

99497), Philadelphia, PA: ASME.

Jung, R., Gasparovic, C., Chavez, R., Flores, R., Smith, S., Caprihan, A., & Yeo. R. (2009).

Biochemical Support for the “Threshold” Theory of Creativity: A Magnetic Resonance

Spectroscopy Study. The Journal of Neuroscience, 29(16), 5319-5325.

Kahneman, D., & Tversky, A. (1972). Subjective Probability: A Judgement of

Representativeness. Cognitive Psychology, 3, 430-454.

Kahneman, D., Tversky, A. (1984). Choices, Values, and Frames. American Psychologist,

39(4), 341-150.

Kan, J., Gero, J., & Tang, H. (2010). Measuring cognitive design activity changes during an

industry team brainstorming session. Design Computing and Cognition’10, Stuttgart,

Germany: Springer.

Koriat, A., Lichtenstein, S., & Fischhoff, B. (1980). Reasons for confidence. Journal of

Experimental Psychology: Human Learning and Memory, 6, 107-118.

83 Kramer, A., Erickson, K., & Colcombe, S. (2006). Exercise, cognition, and the aging brain.

Journal of Applied Physiology, 101, 1237-1242.

Kruger, J., Wirtz, D, Van Boven, L., & Altermatt, W. (2004). The Effort Heuristic. Journal of

Experimental Social Psychology, 40, 91-98.

Kurtoglu, T., Campbell, M., & Linsey, J. (2009). An experimental study on the effects of a

computational design tool on concept generation. Design Studies, 30, 676-703.

Levin, I., Schneider, S., & Gaeth, G. (1998). All Frames Are Not Created Equal: A Typology

and Critical Analysis of Framing Effects. Organizational Behaviour and Human Decision

Processes, 76(2), 149-188.

Linsey, J., Tseng, I., Fu, K., Cagan, J., Wood, K., & Schunn, C. (2010). A study of Design

Fixation, Its Mitigation and Perception in Engineering Design Faculty. Journal of

Mechanical Design, 132(4), 2041003-1-12.

Lippin, R. (2001). Expressive and Creative Arts Therapies, In M. Micozzi, (Ed.), Fundamentals

of Complimentary and Alternative Medicine, pp. 264, Philadelphia, PA: Churchill

Livingstone.

Liu, Y. (2000). Creativity or novelty?: Cognitive-computational versus social-cultural. Design

Studies, 21(3), 261-276.

MacDougall, R. (1905). On the Discrimination of Critical and Creative Attitudes. The Journal

of Philosophy, Psychology and Scientific Methods, 2(11), 287-293.

Malenka, R., & Nicoll, R. (1999). Long-Term Potentiation – A Decade of Progress? Science,

285(5435), 1870-1874.

Marsh, R., Ward, T., & Landau, J. (1999). The inadvertent use of knowledge in a generative

cognitive task. Memory and Cognition, 27(1), 94-105.

Martin, S., Grimwood, P., & Morris, R. (2000). Synaptic Plasticity and Memory: An Evaluation

of the Hypothesis. Annual Review of Neuroscience, 23, 649-711. 84 Martindale, C. (1995). Creativity and Connectionism, In S. Smith, T. Ward & R. Finke (Eds.),

The Creative Cognition Approach, Ch. 11, Cambridge, MA: MIT Press.

Martindale, C. (1999). Biological basis of creativity, In R. Sternberg (Ed.), Handbook of

creativity, Ch. 7, Cambridge, UK: Cambridge University Press.

Martinez, J., & Derrick, B. (1996). Long-Term Potentiation and Learning. Annual Review of

Psychology, 47, 173-203.

Mednick, S. (1962). The Associative Basis of the Creative Process. Psychological Review,

69(3), 220-232.

Mendelsohn, G. (1976). Associative and attentional processes in creative performance. Journal

of Personality, 44, 341-369.

Merriam, S. (2009). Qualitative Research: A Guide to Design and Implementation (3rd ed.), San

Francisco, CA: Josey-Bass.

Meyerowitz, B., & Chaiken, S. (1987). The effect of message framing on breast self-

examination attitudes, intentions, and behavior. Journal of Personality and Social

Psychology, 52(3), 500-510.

Miles, M., & Huberman, A. (1994). Qualitative Data Analysis: An expanded Sourcebook, (2nd

ed.), Thousand Oaks, CA: SAGE Publications Inc.

Mullen, B., & Johnson, C. (1990). Distinctiveness-based illusory correlations and stereotyping:

A meta-analytic integration. British Journal of Social Psychology, 29, 11-28.

National Research Council. (1991). Improving Engineering Design: Designing for Competitive

Advantage. Washington, DC: National Academy Press.

Nickerson, R. (1998). Confirmation Bias: A Ubiquitous Phenomena in Many Guises. Review of

General Psychology, 2(2), 175-220.

85 Nicholl, B., & McLellan, R. (2007). ‘Oh yeah, yeah you got lots of love hearts. The Year 9s are

notorious for love hearts. Everything is love hearts.’ Fixation in Pupils’ Design and

Technology Work (11-16 Years). Design and Technology Education, 12, 34-44.

Osborn, A. (1963). Applied Imagination. New York, NY: Scribner’s.

Perky, C. (1910). An Experimental Study on Imagination. The American Journal of Psychology,

21(3), 422-452.

Perttula, M., & Liikkanen, L. (2006). Structural Tendencies and Exposure Effects in Design

Idea Generation. Proceedings of ASME IDETC/CIE (DETC2006-99123). Philadelphia, PA:

ASME.

Pfenninger, K., & Shubik, V. (2001). Insights into the foundation of creativity: A Synthesis, In

K. Pfenninger & V. Shubik (Eds.), The origins of creativity, pp. 217, Oxford, MA: Oxford

University Press.

Plous, S. (1993). The Psychology of Judgement and Decision Making, New York, NY:

McGraw-Hill.

Purcell, A., & Gero, J. (1996). Design and other types of fixation. Design Studies, 17(4), 363-

383.

Qin, S., van Marle, H., Hermans, E., & Fernandez, G. (2011). Subjective Sense of Memory

Strength and the Objective Amount of Information Accurately Remembered Are related to

Distinct Neural Correlates at Encoding. The Journal of Neuroscience, 31(24), 8920-8927.

Quillian, M. (1962). A revised design for an understanding machine. Mechanical Translation, 7,

17-29.

Reinig, B., Briggs, R., & Nunamaker, J. (2007). On the Measurement of Ideation Quality.

Journal of Management Information Systems, 23(4), 143-161.

Rumelhart, D., Smolensky, P., McClelland, J., & Hinton, G. (1986). Schemata and Sequential

Though Processes in PDP Models, In J. McClelland & D. Rumelhart (Eds.), Parallel 86 Distributed Processing: Explorations in the Microstructure of Cognition, Ch. 14, Cambridge,

MA: MIT Press.

Shrout, P., & Fleiss, J. (1979). Intraclass Correlations: Uses in Assessing Rater Reliability,

Psychological Bulletin, 86(2), 420-428.

Simon, H. (1966). Scientific Discovery and the Psychology of Problem Solving, In R. Colodny

(Ed.), Mind and Cosmos: Essays in contemporary science and philosophy, pp. 22-40,

Pittsburgh, PA: University of Pittsburgh Press.

Smith, S., & Blankenship, S. (1989). Incubation Effects. Bulletin of the Psychonomic Society,

27(4), 311-314.

Stein, M. (1975). Stimulating Creativity, Vols. 1-2, New York, NY: Academic Press.

Sternberg, R., & Lubart, T. (1999). The concept of creativity: Prospects and paradigms, In R.

Sternberg (Ed.), Handbook of Creativity, pp. 3-15, Cambridge, UK: CUP.

Tang, H., & Lee, Y. (2008). Using Design Paradigms to Evaluate the Collaborative Design

Process of Traditional and Digital Media. Design Computing and Cognition’08, Atlanta, GA:

Springer.

Teyler, T., & DiScenna, P. (1987) Long-Term Potentiation. Annual Review of Neuroscience, 10,

131-161.

Thaler, R. (1980). Toward a positive theory of consumer choice. Journal of Economic

Behaviour and Organization, 1, 39-60.

Thompson, R., Berger, T., & Madden, J. (1983). Cellular processes of learning and memory in

the mammalian CNS. Annual Review of Neuroscience, 6, 447-91.

Tversky, A., & Kahneman, D. (1973). Availability: A Heuristic for Judging Frequency and

Probability. Cognitive Psychology, 5, 207-232.

Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice.

Science, 211, 453–458. 87 Tversky, A., & Kahneman, D. (1982). Judgement Under Uncertainty: Heuristics and Biases,

Cambridge, UK: Cambridge University Press.

Van-Praag, H., Christie, B., Sejnowski, T., & Gage, F. (1999). Running enhances neurogenesis,

learning, and long-term potentiation in mice. PNAS, 96(23), 13427-13431.

Vattam, S., Helms, M., & Goel, A. (2008). Compound analogical design: Interaction between

problem decomposition and analogical transfer in biologically inspired design. Design

Computing and Cognition’08, 377-396. Atlanta, GA: Springer.

Viswanathan, V., & Linsey, J. (2011). Design Fixation in Physical Modelling: An Investigation

on the Role of Sunk Cost. Proceedings of ASME IDETC/CIE (DETC2011-47862),

Washington, DC: ASME.

Wagner, U., Gais, S., Haider, H, Verleger, R., & Born, J. (2004). Sleep inspires insight. Nature,

427, 352-254.

Wallas, G. (1926) The Art of Thought, New York, NY: Harcourt, Brace and Company.

Ward, T. (1994). Structured Imagination: The Role of Category Structure in Exemplar

Generation. Cognitive Psychology, 27, 1-40.

Ward, T. (1995). What’s Old About New Ideas? In S. Smith, T. Ward & R. Finke (Eds.), The

Creative Cognition Approach, Ch. 7, Cambridge, MA: MIT Press.

Wason, P. (1968). Reasoning about a rule. Quarterly Journal of Experimental Psychology,

20(3), 273-281.

Wickens, C., Lee, J., Liu, & Gordon Becker, S. (2004). Design and Evaluation Methods. Human

Factors Engineering, Chapter 3, Upper Saddle River, NJ: Pearson Prentice Hall.

Young, J. (2007, March 6). Neuron to Glial Synapse on Axons? Retrieved from

http://scienceblogs.com/purepedantry/2007/03/neuron_to_glia_synapse_on_axon.phyhp.

88 Appendix A: Demographic Questionnaire

Instructions: The purpose of this questionnaire is to assess your general background and gather some information relevant to your ability to perform tasks related to the study. Your personal identity will not be associated with any of your responses and this information will be kept confidential.

1. Gender:  Male  Female 2. Age: ______3. Specified major (area of study): ______4. Current level of study:  Graduate (or)  Undergraduate 5. Year of program: ______6. Do you have previous design experience?  Yes  No a. How many years of design experience do you have? _____ b. Please list any relevant design courses or projects you have been involved in: i. ______ii. ______iii. ______iv. ______v. ______7. Is English your first language?  Yes  No a. Please rate your English reading fluency on a scale of 1 - 5 (circle one).

1 2 3 4 5 Very Poor Poor Average Good Excellent

Thank you for filling out this questionnaire

89 Appendix B.1: Design Problem Before Defixation Task

Design a system that automatically administers water to a house plant. The system must provide a potted plant with a predetermined amount of water every week. You must consider how the flow of water will be controlled, how the water will be administered to the plant, what the water source will be, and what will power your system.

Example Solution

A timer valve has been attached to a house water main. At predetermined intervals, the valve opens and allows the desired amount of water to flow through a showerhead onto the plant.

Design Solutions

90 Appendix B.2: Design Problem After Defixation Task

Design Problem You will now have 10 additional minutes to continue working on the previously presented design problem. Try and generate as many distinct and feasible ideas as possible. If you have any questions, please ask the researcher and he will be happy to assist you. Please sketch your design solutions on the bottom and back of this page, additional pages have been provided if required. You are free to include descriptions of your design solution with the sketches.

Design Solutions

91 Appendix C.1: Defixation Task Instructions

TREATMENT GROUP INSTRUCTIONS Design problems like the previous one are typically more analytical in nature and involve activation of the right hemisphere of the brain. To stimulate the left hemisphere of the brain we will be asking you to engage in a language based learning task, as this is typically associated with activation of the left hemisphere of the brain. Physical activity has also been shown to enhance blood flow in the brain, so exercise will be incorporated the task.

On the following page is a story designed to help you learn beginner’s Swedish. Please read through the story and attempt to learn the English equivalent of the Swedish words and phrases used in the story. During the activity you may not write anything down, and may not externally vocalize the words or phrases. Instead simply read and internally verbalize the material to assist your learning. You will be given 10 minutes to complete the language learning portion of the experiment.

In addition, you are being asked to perform a mild aerobic step exercise for the duration of the language task. Please step up and down from the step block provided while completing the language activity. Keep the pace of stepping brisk and constant as if you were walking up a flight of stairs. If you become fatigued and cannot continue the exercise, you may stop.

Please devote your full attention to the language task as you will be tested on the material covered in the language exercise. The test will consist of translating the common Swedish words and phrases presented in the story into English. You will also be tested on the content of the story.

______

CONTROL GROUP INSTRUCTIONS Design problems like the previous one are typically more analytical in nature and involve activation of the right hemisphere of the brain. To stimulate the left hemisphere of the brain we will be asking you to engage in a language based task, as this is typically associated with activation of the left hemisphere of the brain.

92 Appendix C.2: Defixation Task Story Material

Mike is visiting his friend Greger in Sweden. He is from Toronto, but has learned some basic Swedish over the past couple of days, so he decides to go out for lunch and see how well he can get by on his own.

A Simple Greeting Mike is taking a bus to get to a restaurant and decides to practice his Swedish with a stranger next to him at the bus stop.

Mike: God morgon! (Good morning!) Jag heter Mike, (My name is Mike,) Vag heter du? (what is your name?)

Stranger: Roligt att träffas Mike, (Nice to meet you Mike,) Jag heter Henrik. (my name is Henrik.)

Mike: Rolligt att träffas Henrik, (Nice to meet you Henrik,) var kommer du ifrån? (where are you from?)

Stranger: Jag är från Stockholm, (I am from Stockholm) Var kommer du ifrån? (Where are you from?)

Mike: Jag är från Toronto (I am from Toronto) och jag är ingenjör, (and I am an engineer,) Var arbetar du med? (What is your profession?)

Stranger: Jar är lärare. (I am a teacher.)

Mike: Talar du engleska? (Do you speak English?)

Stranger: Ja, lite. (Yes, a little)

Mike’s bus comes

Mike: Adjö Henrik. (Goodbye Henrik)

At the Restaurant Mike arrives at the restaurant and takes a seat at a table. The waiter approaches him.

Waiter: God middag, (Good afternoon,) jag heter Sven. (my name is Sven.) Vad kan jag få för dig? (What can I get for you?)

Mike: Kan jag få en koppe kaffe? (May I have a cup of coffee)

Waiter: Ja, vill du något att äta? (Yes, would you like anything to eat?)

Mike: Jag skulle vilja ha (I would like to have) en kyckling smörgås (a chicken sandwich) och pomme frites, vänligen. (and French fries, please).

93 Waiter: Din mat kommer att vara klar (Your food will be ready) i femton minuter. (in 15 minutes)

20 minutes later

Waiter: Hära är din mat, (Here is your food,) ledsen för vänta. (sorry for the wait.)

Mike: Det är bra, tack. (That’s fine, thank you.)

Mike finishes his lunch.

Waiter: Hur van din mat? (How was your food?)

Mike: Bra tack. Kan jag få notan? (Good, thank you. May I have the bill?)

Waiter: Naturligtvis, (Of course,) här är. (here you are)

Getting Lost Mike decides to walk back to his friend’s house from the restaurant, but can’t remember which way to go. After struggling with street signs and a map for a few hours he decides to ask a friendly stranger.

Mike: God middag, talar du engelska? (Good afternoon, do you speak English?)

Stranger: Nej, tyvärr. (No, sorry.) Kan jag hjälpa dig? (Can I help you?)

Mike: Jag är vilse. (I am lost.) Jag letar efter femton Råstavägen gata. (I am looking for fifteen Råstavägen street)

Stranger: Det är sex gator norr, (It is six streets north,) jag kan gå med dig. (I can walk with you.)

Mike: Tack så mycket! (Thank you very much!)

Mike Gets Home

Greger: Hur van din dag? (How was your day?)

Mike: Det var bra. (It was fine). Jag tog bussen, åt lunch, och gick hem. (I took the bus, had lunch, and walked home.)

Greger: Hade du några problem? (Did you have any troubles?)

Mike: Naturligtvis inte! (Of course not!)

94 Appendix D: Recall Test for Defixation Material

This portion of the experiment is meant to test your recall for the information presented in the language task.

Section 1 In this first section, you will be asked to recall the English translation of some Swedish words or phrases presented in the story. Next to each Swedish word or phrase, please write the English translation as accurately as possible.

1) Jag heter Sven. ______

2) Jag är ingenjör. ______

3) Lärare ______

4) Talar du engleska? ______

5) Adjö ______

6) God middag ______

7) Kan jag få en koppe kaffe? ______

8) Kyckling smörgås ______

9) Femton ______

10) Hur van din mat? ______

11) Kan jag få notan? ______

12) Jag är vilse. ______

13) Kan jag hjälpa dig? ______

14) Tack så mycket! ______

15) Hur van din dag? ______

95 Section 2

Please answer the following questions (in English) about the content of the story you read previously.

1) What was the name of the stranger Mike was talking to at the bus stop? ______

2) What was the profession of the stranger at the bus stop? ______

3) Where was the stranger at the bus stop from? ______

4) What did Mike order for lunch at the restaurant to eat? ______

5) What did Mike order at the restaurant to drink? ______

6) What was the house number of Mike’s friend’s house? ______

7) In what direction, and how many blocks did the stranger walk Mike to take him back to his friend’s house? ______

96 Appendix E: Task Difficulty Questionnaire

The purpose of this questionnaire is to assess the subjective level of difficulty you experienced with the tasks in the study.

For each question, circle the number (1-7) that corresponds most accurately with your perception of the tasks difficulty.

1) I felt that the level of physical exertion required of me in this study was:

1 ------2 ------3 ------4 ------5 ------6 ------7 Extremely Low Moderate Extremely High

2) I felt that learning and recalling the Swedish story and phrases was:

1 ------2 ------3 ------4 ------5 ------6 ------7 Very Easy Moderate Very Difficult

3) I felt that generating design solutions the first time I worked on the design problem was:

1 ------2 ------3 ------4 ------5 ------6 ------7 Very Easy Moderate Very Difficult

4) I felt that generating design solutions the first time I worked on the design problem was:

1 ------2 ------3 ------4 ------5 ------6 ------7 Very Easy Moderate Very Difficult

97 Appendix F: Fixation Coding Instructions

You have been given a set of design solutions that participants generated in response to a problem requiring the design of a system that automatically waters a houseplant (see attached). Your task is to score each design solution to determine an objective fixation score. The scoring is based on four categories: 1) Water Source, 2) Water regulation, 3) Water Transfer, and 4) Energy Source.

On the scoring sheets attached, assign a 1 to the category if it matches the example solution (fixation) or a 0 to the category if it is not the same as the example solution. In cases where an element is not completely distinct from the example solution, or only slightly different you may assign half points (0.5). (e.g. 1 = fixated, 0.5 = somewhat fixated, 0 = no fixation). In the overall column please evaluate the similarity of the solution to the fixation example from 1-7 (1 = very similar, 4 = neither similar or dissimilar, 7 = not at all similar) using a holistic evaluation approach taking into consideration similarity in function, concept, and design.

Design solutions have been randomly ordered and numbered. Please ensure the number of the design solution you are scoring matches the design solution number on the scoring sheet. The design solution numbers do not convey any temporal ordering. If you have any comments relevant to your scoring describe them in the space provided or on the back of the sheet, but make it clear what solution the comment corresponds to. Solutions have been grouped by participant, however the ordering (before or after the defixation activity) has been randomized. Therefore you will be viewing one participant’s set of designs before moving onto the next participant’s. If you see a participant repeat an idea make a note of it in comments (i.e. “repeat”).

Example Solution for Comparison: In this solution, a timer valve has been attached to a house water line. Once a day the timer opens and the valve and allows exactly 1/70 of a litre of water to be released. The water is administered to the house plant through a spray nozzle.

The Four Categories 1) Water Source – Municipal Water Supply (Household) 2) Water regulation – Timer valve 3) Water Transfer – Pressurized release through house pipes and a sprinkler head 4) Energy Source – Municipal Electricity

98 Appendix G: Ratersʼ Raw Scores for Fixation Coding

RATER 1 Concept Participant Source Regulation Transfer Energy Total Overall

1 1 0.5 1 0.5 1 3 2 2 1 0.5 0 0 0.5 1 6 3 1 0 0 0 0 0 7 4 1 0 0 0 0.5 0.5 7 5 1 0.5 0.5 1 0 2 3 6 2 1 1 0.5 1 3.5 1 7 2 1 1 0.5 1 3.5 1 8 3 1 1 1 1 4 1 9 3 0.5 0.5 0.5 0.5 2 3 10 3 1 0 1 0.5 2.5 3 11 3 1 0.5 1 1 3.5 1 12 3 1 0.5 1 1 3.5 1 13 4 0 0 0 0 0 7 14 4 0 0 0 0 0 7 15 4 0.5 0 0 0.5 1 6 16 4 0.5 0 0.5 0.5 1.5 4 17 4 0 0 0 0.5 0.5 7 18 4 0 0 0 0 0 7 19 5 0.5 0 0.5 0.5 1.5 4 20 5 1 1 1 1 4 1 21 5 0.5 1 0.5 1 3 2 22 5 1 1 1 1 4 1 23 5 0.5 1 0.5 1 3 2 24 5 0 0 0 0.5 0.5 7 25 5 1 1 0.5 1 3.5 1 26 6 0 0 0 0 0 0 27 6 0 0 0 0 0 0 28 7 1 0.5 1 1 3.5 1 29 7 0 0.5 0 0.5 1 6 30 7 1 0 0.5 1 2.5 4 31 7 1 0.5 1 1 3.5 1 32 7 0 0.5 0 0.5 1 6 33 7 1 0 0.5 1 2.5 4 34 7 0.5 0 0.5 0.5 1.5 4 35 8 0.5 0 0 0 0.5 6 36 8 1 0.5 1 1 3.5 1 37 8 1 0 0 0.5 1.5 4 38 8 1 0 0 0.5 1.5 4 39 8 0.5 1 0.5 1 3 3 40 9 1 0 0.5 1 2.5 3 41 9 0 0 0 0 0 7 42 9 0 0 0 0 0 7 43 9 0.5 0 0 0.5 1 5 44 9 1 0 0 0.5 1.5 4

99 45 9 0.5 1 0 1 2.5 3 46 9 0.5 0 0 0 0.5 7 47 9 0 0 0 0 0 7 48 10 1 1 0.5 0 2.5 2 49 10 1 1 0.5 1 3.5 1 50 11 1 1 0.5 1 3.5 2 51 11 0.5 1 1 1 3.5 1 52 11 1 1 1 1 4 1 53 11 1 1 1 0.5 3.5 1 54 11 0.5 0 0.5 0 1 7 55 12 1 1 0.5 1 3.5 2 56 12 0 0 0 0 0 7 57 12 0.5 1 0.5 1 3 2 58 12 1 0 0.5 0.5 2 3 59 12 0.5 0.5 0.5 1 2.5 2 60 12 0.5 0 0.5 0 1 7 61 12 0 0 0 0 0 7 62 13 0.5 0 0.5 0 1 7 63 13 1 0 0.5 1 2.5 5 64 13 0 0 0 0 0 0 65 13 1 0 0.5 1 2.5 5 66 13 0 0.5 0 0 0.5 7 67 13 0 0 0 0.5 0.5 7 68 13 0 0 0 0 0 7 69 14 0.5 0 1 0.5 2 3 70 14 0.5 0 0 0.5 1 6 71 14 1 0 0 0.5 1.5 6 72 14 1 1 1 1 4 1 73 14 1 0.5 1 1 3.5 1 74 14 0.5 0.5 1 1 3 2 75 14 1 0.5 1 1 3.5 1 76 15 1 1 0.5 1 3.5 1 77 15 1 0 0.5 1 2.5 5 78 15 1 0 0.5 1 2.5 5 79 15 0 0 0 0 0 7 80 15 1 0.5 0 1 2.5 5 81 16 1 1 1 1 4 1 82 16 1 1 0.5 1 3.5 2 83 16 0.5 1 1 1 3.5 2 84 16 1 1 0.5 1 3.5 1 85 17 0.5 0 0 0 0.5 7 86 17 0 0.5 0 0 0.5 7 87 17 1 0 0.5 1 2.5 4 88 17 1 0.5 0.5 0.5 2.5 3 89 17 0 0 0 0 0 7 90 18 0.5 0.5 0.5 1 2.5 3 91 18 0.5 0.5 1 1 3 3 92 18 0.5 0 0 0 0.5 7 93 18 0 0 0 0 0 7 100 94 18 1 0.5 1 1 3.5 1 95 18 1 1 1 1 4 1 96 19 1 0.5 1 1 3.5 1 97 19 1 0.5 1 1 3.5 1 98 19 1 1 1 1 4 1 99 20 1 0.5 0 1 2.5 3 100 20 0 0 0 0 0 7 101 20 0 0 0.5 0 0.5 6 102 20 0 0 0 0 0 7 103 20 0 0 0 0 0 7 104 20 0.5 0 0 0.5 1 6 105 20 0 0 0 0 0 7 106 20 0 0 0 0 0 7 107 21 1 1 0 1 3 2 108 21 0 0.5 0 1 1.5 5 109 21 0.5 1 0.5 1 3 2 110 21 0 1 1 1 3 3 111 21 0 0 0 0 0 0 112 21 1 0 0 0.5 1.5 7 113 22 0.5 0 0 0 0.5 6 114 22 0 0.5 0 0 0.5 7 115 23 0.5 0 0.5 0 1 5 116 23 0 0 0 0 0 7 117 24 1 0.5 1 1 3.5 1 118 24 0.5 0 0 0.5 1 7 119 24 0.5 0.5 0.5 0.5 2 3 120 24 0 0 0 0 0 7 121 25 0.5 1 0.5 0 2 3 122 25 0.5 1 0 0 1.5 4 123 25 0.5 0.5 0.5 0.5 2 3

101

RATER 2 Concept Participant Source Regulation Transfer Energy Total Overall 1 1 1 1 0 1 3 2 2 1 1 0.5 0 1 2.5 3 3 1 1 1 0 1 3 3 4 1 0 1 0.5 1 2.5 4 5 1 1 1 1 0 3 2 6 2 1 1 0.5 1 3.5 1 7 2 1 1 0.5 1 3.5 1 8 3 1 1 1 1 4 1 9 3 1 0 0.5 0 1.5 5 10 3 1 0 1 0.5 2.5 5 11 3 1 0.5 1 0.5 3 3 12 3 1 0.5 1 1 3.5 1 13 4 0 0 0 0.5 0.5 7 14 4 0 0 0 0.5 0.5 6 15 4 1 0 0.5 0.5 2 5 16 4 1 0 0 1 2 3 17 4 1 0 0 1 2 4 18 4 0 0 0.5 0 0.5 6 19 5 1 0 0.5 0.5 2 5 20 5 1 1 1 1 4 1 21 5 0.5 1 1 1 3.5 2 22 5 1 1 1 1 4 1 23 5 1 1 0.5 1 3.5 1 24 5 0 0 0 1 1 6 25 5 1 1 0.5 1 3.5 2 26 6 0 0 0 0 0 0 27 6 0 0 0 0 0 0 28 7 1 1 1 1 4 1 29 7 0 1 1 1 3 2 30 7 1 0 1 1 3 2 31 7 1 1 1 1 4 1 32 7 0 1 0 1 2 2 33 7 1 0 0.5 0.5 2 2 34 7 1 0 1 0.5 2.5 2 35 8 0.5 0 0.5 0 1 5 36 8 1 0.5 1 1 3.5 6 37 8 1 0 0 1 2 5 38 8 1 0 0.5 0 1.5 6 39 8 0 0 0.5 1 1.5 3 40 9 1 0 0.5 1 2.5 3 41 9 0 0 0 0 0 7 42 9 0 0 0.5 0 0.5 6 43 9 1 1 0.5 1 3.5 2 44 9 1 0 0.5 0.5 2 3 45 9 0 1 1 0 2 2 102 46 9 0 0 0 0 0 7 47 9 0 0 0 0 0 7 48 10 1 1 1 1 4 1 49 10 1 0.5 0.5 1 3 2 50 11 1 1 1 1 4 1 51 11 0.5 1 1 1 3.5 2 52 11 0.5 1 1 1 3.5 1 53 11 1 1 1 1 4 1 54 11 0.5 0 0 0.5 1 7 55 12 1 1 0.5 1 3.5 2 56 12 1 0 0 0 1 6 57 12 0.5 1 0.5 1 3 3 58 12 1 0 0 0 1 6 59 12 0.5 1 1 1 3.5 1 60 12 0.5 0 0 0.5 1 4 61 12 0 0 0 0 0 7 62 13 0 0 0 0 0 7 63 13 1 0 0 0 1 3 64 13 0 0 0 0 0 0 65 13 0.5 0 0 1 1.5 5 66 13 0 0 0 0 0 7 67 13 0 0 0 0 0 7 68 13 0 0 0 0 0 7 69 14 0 0 1 0 1 6 70 14 0 0 0 0 0 7 71 14 1 0 0 0 1 6 72 14 1 0 1 1 3 2 73 14 1 0 1 1 3 2 74 14 1 0 1 1 3 2 75 14 1 0 1 1 3 2 76 15 1 1 1 1 4 1 77 15 1 0 0 0 1 5 78 15 1 0 0 0 1 5 79 15 0 0 0 0 0 7 80 15 1 0.5 0 1 2.5 3 81 16 1 1 1 1 4 1 82 16 1 1 0 1 3 2 83 16 0.5 1 1 1 3.5 1 84 16 1 1 0.5 1 3.5 2 85 17 1 1 0 1 3 2 86 17 0 0.5 0 1 1.5 5 87 17 1 0 1 1 3 2 88 17 1 0 1 1 3 3 89 17 0 0 0 1 1 5 90 18 0 1 1 1 3 3 91 18 1 0 0 1 2 3 92 18 1 0 0.5 0 1.5 3 93 18 0.5 0 0.5 0 1 6 94 18 1 0.5 1 1 3.5 2 103 95 18 0 0 0 0 0 0 96 19 1 0.5 1 1 3.5 2 97 19 1 0.5 1 1 3.5 2 98 19 1 0.5 1 1 3.5 1 99 20 1 1 0 1 3 3 100 20 1 0 0.5 1 2.5 3 101 20 1 0 0.5 1 2.5 3 102 20 0 0 0 0 0 0 103 20 0 0.5 0 0 0.5 6 104 20 1 0 0 1 2 3 105 20 0 0 0.5 1 1.5 3 106 20 1 0 0 1 2 5 107 21 1 1 0 1 3 2 108 21 0 1 0.5 1 2.5 4 109 21 0 0 1 1 2 3 110 21 0 1 1 1 3 2 111 21 0 1 1 1 3 2 112 21 1 0 0 0 1 5 113 22 0.5 0.5 0.5 1 2.5 3 114 22 0 0.5 0 1 1.5 5 115 23 1 0 0.5 1 2.5 5 116 23 0 0 0 0 0 7 117 24 1 1 1 1 4 1 118 24 1 1 0.5 1 3.5 3 119 24 1 1 0.5 1 3.5 3 120 24 0.5 1 0.5 1 3 3 121 25 1 0.5 0.5 1 3 3 122 25 1 1 0.5 1 3.5 2 123 25 1 1 1 1 4 1

104

Appendix H.1: Participant Concepts Ranked Low in Fixation Before Defixation

Subject 20 – Concept Numbers: 99, 100, 101 (Average Fixation Score 0.46/1.00)

105

Appendix H.1 – Participant Concepts Ranked Low in Fixation After Defixation

Subject 20 – Concept Numbers: 102, 103, 104, 105 (Average Fixation Score 0.22/1.00)

106

Appendix H.2: Participant Concepts Ranked High in Fixation Before Defixation

Subject 16 – Concept Numbers: 81, 82 (Average Fixation Score 0.91/1.00)

107

Appendix H.2: Participant Concepts Ranked High in Fixation After Defixation

Subject 16 – Concept Numbers: 83, 84 (Average Fixation Score 0.88/1.00)

108 Appendix I: Design Problems and Biological Analogies

1. Promotional Mailing Problem

You are a marketing director for a credit card company. You are looking for an effective strategy to distribute sign-up promotional mailings within a city. You would like to distribute promotional mail to selected neighborhoods in the city so that a large proportion of the promotional mail actually result in people signing up. In other words, you don’t want to waste resources on sending promotional mail to neighborhoods where people are not likely to sign up.

Assuming that you don’t have any demographic information of the city, how would you optimize the use of promotional mailings?

Biological Phenomenon (Ant)

An ant colony can identify the shortest path between its nest and food source with the following strategy. Ants depart the colony to search randomly for food, laying down pheromones on the trail as they go. When an ant finds food, it follows its pheromone trail back to the nest, laying down another pheromone trail on the way. Pheromones have more time to dissipate on longer paths, and less time to dissipate on shorter paths. Shorter paths are also travelled more often relative to longer paths, so pheromones are laid down more frequently on shorter paths.

Additional ants follow the strongest pheromone trails between the food source and the nest, further reinforcing the pheromone strength of the shortest path.

2. Authorized Disassembly Problem

Original equipment manufacturers (OEM’s) want easy disassembly of their products to reduce disassembly cost and increase the net profit from reuse and recycling at product end of life.

However, OEM’s are also concerned with protecting high-value components from theft and access by competitors. How can you allow disassembly that is easy but only by those authorized? 109 Biological Phenomenon (Enzymes)

Enzymes are complex proteins that bind to specific substrates (molecules) and form enzyme- substrate complexes that perform biochemical activities. The specific binding is achieved when the active site of an enzyme geometrically matches its corresponding substrate. However, an enzyme changes its shape with environmental factors such as pH and temperature. This shape change alters the conformation of the enzyme’s active site to the point where substrates can no longer fit, thereby disabling the function of the enzyme-substrate complex.

3. Wet Scrubber Problem

Wet scrubbers are air pollution control devices that remove pollutants from industrial exhaust systems. In conventional wet scrubbers, exhaust gas is brought into contact with a liquid solution that removes pollutants from the gas by dissolving or absorbing them into the liquid.

The removal efficiency of pollutants is often improved by increasing the contact time or the contact area between the exhaust gas and the scrubber liquid solution. What other strategy could be used to increase the removal efficiency of wet scrubbers?

Biological Phenomenon (Penguins)

Penguins are warm blooded yet keep their un-insulated feet at a temperature close to freezing to minimize heat transfer to the environment. The veins that carry cold blood from the feet back to the body are located closely to the arteries that carry warm blood from the body to the feet. The warm blood flows in the opposite direction as the cold blood, which allows the penguins to transfer the most heat to the cold blood. This reduces both the amount the returning blood can drop the core body temperature, and the amount of heat lost through the feet.

110 Appendix J – Coded Verbal Protocol for Group 2

111

112

113

114

115

116

117 Appendix K: Confirmation and Disconfirmation Coding Scheme

You have been given a set of notes that participants generated while evaluating a design belief. Your task is to review the notes in their entirety and determine: 1) How many pieces of evidence/arguments are evaluated (Cases) with respect to the belief, 2) How many of those cases are evaluated in order to confirm the belief, and 3) How many of those cases are evaluated in order to disconfirm the belief. Please see Problem 2 for a review of the problem as it was presented to participants, the design belief that they were evaluating, and the concepts they evaluated.

On the scoring sheets attached, record the participant number, the total number of cases evaluated, the number of confirmatory cases evaluated, the number of disconfirmatory cases evaluated, and the number of cases that were ambiguous or could not be coded. To determine what constitutes a “case” from the notes, determine what sentences, points or arguments, constitute clauses that are independent of each other. If a clause can stand-alone and is used to support or refute the belief in question, it should be considered as an individual case. If the same clause is applied multiple times (X), than it can be considered as X number of cases. For information in the form of a decision matrix, each piece of evidence evaluated should have been included in the left-most column. If that evidence is applied multiple times (X), it too can be considered as X number of cases.

The participant numbers have been randomly assigned and do not convey any temporal or categorical ordering. If you have any comments relevant to your rating describe them in the space provided or on the back of the sheet, but make it clear what participant and case the comment corresponds to.

118 Coding Sheet

Participant Total No. of Confirmatory Cases Disconfirmatory Ambiguous Comments No. Cases Cases Cases

119 Appendix L: Confirmation Bias Problem 1

Problem 1: The Washing Machines

A common belief among consumers is that washing machines that are more water efficient (use less water per load) are also more energy efficient (use less energy per load). You would like to know whether this belief is true or false. A local appliance store has said it can send you manufacturer specifications for some of their machines. The store has four different models, but they can only send you the information on two of the four. However, the store was able to tell you a little about each model with respect to relatively how water or energy efficient it is to help you make your decision.

Your task is to select two of the four machines that you believe will be the most useful in evaluating the validity of the following belief:

“Washing machines that are highly water efficient are also highly energy efficient.”

Please record your choice, and any relevant considerations that informed your choice, on the blank sheet provided.

120

A B

C D

121 Appendix M: Confirmation Bias Problem 2

Problem 2: Fixation in Design

Fixation There has been a significant amount of research demonstrating that designers often become fixated by examples of successful design solutions. The research indicates that when designers see an example solution for a design problem they are working on, they often incorporate elements of that example into their own design solutions. This effect has been observed even when designers are instructed not to fixate on examples, and even among experienced designers.

An experiment was run to test the hypothesis that designers fixate on examples. The design problem and example solution given to participants is seen at the bottom of the page. Six participants generated solutions for the problem; their concepts can be seen on the next page.

Your job is to look at the results of the experiment (the participant concepts) to evaluate the validity of the fixation hypothesis, stated below:

“The presence of an example solution causes designers to fixate and incorporate elements of the example into their own solutions.”

Design Problem Design a system that automatically administers water to a house plant. The system must provide a potted plant with a predetermined amount of water for a predetermined amount of time.

Example Solution

A timer valve has been attached to a house water main. At predetermined intervals, the valve opens and allows the desired amount of water to flow through a sprinkler head onto the plant.

122 Concepts to Evaluate

A water tank is filled A water wheel sits in a tank Rainwater is collected from a water line. A filled by a house water line. in a tank. A timer valve ball-float is attached to The rotational speed of the is set to release water a valve at the bottom of wheel is programmed so that from the tank onto the the tank. The float and it pours the desired amount plant at predetermined valve are set so that of water into the plant. intervals based on the the desired amount of desired water volume. water is released from the tank onto the plant.

A drip container is The plant is placed Sponges are inserted into the soil. on a platform that hydrated and The container is lowers into and out placed on the soil filled with enough of a tank of water. around the plant. water, and drips at a The frequency and The number of rate, so the plant duration is pre-set to sponges is receives the desired provide adequate adjusted to amount of water. water. provide enough hydration.

123 Appendix N.1 – Treatment Group Instructions for Concept Evaluation

Instructions

Your task is to evaluate the belief stated in Problem 2. You have been provided with 6-design concepts, which are relevant pieces of evidence you may use to help you evaluate that belief. You have also been provided with procedural instructions to help formalize the process of decision-making. Please use the procedure to help you evaluate the belief provided in Problem 2. Read through the procedure completely before beginning.

Procedure

1) Identify all the possible conclusions you could draw, e.g. what are the options you can decide between with respect to the belief provided. a. Example: If you are evaluating the belief smoking causes cancer, you could decide that the belief is true, or false, conditionally true, etc. 2) Consider all the evidence available that is relevant in deciding to reject or accept each possible conclusion. In this case that evidence will come from your evaluation of the 6 design concepts. a. Example: In evaluating the belief from step 1, one piece of evidence could be “results of studies examining a correlation between smoking and lung cancer”. A different piece of evidence could be “anecdotal cases of individuals who have smoked their whole lives and not been diagnosed with cancer”. 3) Prepare a table (see template below). Place each conclusion (from step 1) in its own cell across the top row. Put each piece of evidence (from step 2) in its own cell in the left column. 4) Work through the table and evaluate each conclusion relative to each piece of evidence. a. Example: Given the evidence “results of studies examining a correlation between smoking and lung cancer” you would evaluate each of your conclusions regarding the smoking- cancer belief. This evidence would likely verify the conclusion that the belief is true, however it would contradict the conclusion that the same belief is false. b. Consider the value of each piece of evidence, how strongly or weakly does it verify or contradict a conclusion. 5) Finally select the most likely or favourable conclusion, e.g. what is your conclusion regarding the belief stated in Problem 2 given your evaluation of the evidence.

Table Template

Conclusion1 Conclusion2 Conclusion…

Evaluation of Conclusion1 Evidence1 … … given Evidence1

Evaluation of Conclusion2 Evidence2 … … given Evidence2

Evaluation of Conclusion… Evidence… … … given Evidence…

Please use the blank page provided to generate your MACH table. Please use a separate blank sheet for any other decision-making considerations you make that aren’t integrated into the MACH table. We estimate that this task will take approximately 15 minutes, however there is no time limit.

124 Appendix N.2 – Control Group Instructions for Concept Evaluation

Instructions

Your task is to evaluate the belief stated in Problem 2. You have been provided with 6-design concepts which are relevant pieces of evidence you may use to help you evaluate that belief. Please review those design concepts carefully, and provide a written record that reflects the considerations you made in reaching your conclusion.

Please make sure to write down anything that you believe was relevant in helping you evaluate the belief from Problem 2 based on the evidence (6 concepts) that was provided. Record these considerations in point form notes as they occur to you.

Once you feel you have performed a thorough analysis please write down your opinion regarding the belief stated in Problem 2. We estimate that this task will take approximately 15 minutes, however there is no time limit.

125 Appendix O.1 – Example of Participant Concept Evaluation Matrix

126 Appendix O.2 – Example of Participant Concept Evaluation Notes

127