Towards a Theoretical Understanding of Creative

The Pennsylvania State University

The Graduate School

College of Engineering

TOWARDS A THEORETICAL UNDERSTANDING OF CREATIVE

CONCEPT SELECTION IN ENGINEERING DESIGN

A Dissertation in

Industrial Engineering

Christine Toh

Submitted in Partial Fulfillment of the Requirements for the Degree of

Doctor of Philosophy

August 2016

The dissertation of Christine Toh was reviewed and approved* by the following:

Scarlett R. Miller Assistant Professor of Engineering Design and Industrial Engineering Dissertation Advisor Chair of Committee

Gül E. Okudan Kremer Professor of Engineering Design and Industrial Engineering Major Program Member of Committee

Timothy W. Simpson Professor of Mechanical Engineering and Industrial Engineering Major Program Member of Committee

Samuel T. Hunter Associate Professor of Psychology Outside Field/ Unit Member of Committee

Janis Terpenny Professor of Industrial Engineering Peter and Angela Dal Pezzo Department Head of Industrial Engineering

*Signatures are on file in the Graduate School

ii ABSTRACT

There is a growing need for companies to generate innovative solutions and products in order to stay competitive in today’s constantly shifting engineering and technology landscape. Rapid changes in customer needs, emerging technologies, and new market opportunities demand quick and effective product design processes that can provide a competitive advantage. The focus on improving the effectiveness of the design process has led to renewed attention on studying and improving creativity training in engineering education. Since creative solutions contribute the most value to the design process, design educators are recognizing that students graduating engineering programs need to be trained for creativity prior to entering the workforce. While a large focus of creativity training in engineering education has been on helping students develop creative ideas, researchers have argued that the “availability of creative ideas is a necessary but insufficient condition for innovation”. This is because creative ideas are often filtered out during the design process because of people’s inadvertent bias against creativity. In other words, concept selection can now be seen as the “gatekeeper of creativity.” In addition, informal selection methods, often used in engineering industry, are subject to decision-making biases such as ownership bias and risk aversion. However, few studies have explored the impact of these factors on creative concept selection in engineering design education leaving it unclear exactly how these factors impact the design training of the next generation of design engineers. Therefore, the objective of this dissertation was to develop a theoretical understanding of the individual factors that impact creative concept selection processes in engineering design education. This was achieved through empirical studies with 427 engineering design students, spanning 3 research objectives: (1) an exploration of the decision-making process in informal student team concept selection practices, (2) a detailed investigation of the cognitive biases and personal attributes that influence team decision-making and creativity, and (3) identification of the underlying constructs that influence individual preferences for creative concepts and its influence on engineering student behavior. The results of this research contribute fundamental knowledge on the factors and elements that constitute preference for creativity in engineering student decision- making, allowing researchers to develop tools and methods that encourage creativity, improve innovation effectiveness, and increase the competitive advantage of the design process.

iii TABLE OF CONTENTS

List of Figures……………...... …………………..……………………….....…...... viii List of Tables…..……………………………………...... ………...... ………………...... x Dissertation Publications...... xi Acknowledgements...... xii

1 INTRODUCTION ...... 2

1.1 | Research Objectives and Significance ...... 4 1.2 | Expected Contributions ...... 6 1.3 | Document Outline ...... 7

2 LITERATURE REVIEW: THE CONCEPT SELECTION PROCESS IN DESIGN ...... 8

2.1 | The Design Process ...... 9 2.2 | Supporting Concept Selection in Design ...... 11 2.3 | Team Interactions and Processes in Design ...... 14 2.4 | The Impact of Individual Biases and Attributes on Concept Selection ...... 15 2.4.1 | Bias Against Creativity ...... 17 2.4.2 | Risk and Ambiguity Aversion ...... 18 2.4.3 | Personality Traits ...... 19 2.4.4 | Ownership Bias ...... 21 2.5 | Opportunities for Investigation ...... 23 3 HOW ENGINEERING STUDENT TEAMS SELECT DESIGN CONCEPTS ...... 25

3.1 | Methodology ...... 26 3.1.1 | Participants ...... 26 3.1.2 | Brainstorming Activity ...... 26 3.1.3 | Concept Selection Activity ...... 28 3.1.4 | Quantitative Data Metrics ...... 30 3.1.5 | Qualitative Data Coding Procedure ...... 32 3.2 | Data Analysis and Results ...... 33 3.2.1 | Discussion Topics During Team Concept Selection ...... 33 3.2.2 | The Impact of Propensity of Creative Concept Selection on the Frequency of Discussion Topics ...... 40 3.3 | Chapter Discussion and Summary ...... 44

iv 4 RISK ATTITUDES AND PERSONALITY TRAITS IN CREATIVE CONCEPT SELECTION ...... 47

4.1 | Methodology ...... 48 4.1.1 | Participants ...... 49 4.1.2 | Procedure ...... 49 4.1.3 | Metrics ...... 50 Creativity Metrics ...... 50 Risk and Ambiguity Aversion Metrics ...... 52 Big 5 Factors of Personality Trait Metrics ...... 55 4.2 | Data Analysis and Results ...... 55 4.2.1 | Do Creative ideas have a higher likelihood of being selected during concept selection? ...... 57 4.2.2 | Does Creative idea generation ability relate to the teams’ propensity for creative concept selection? ...... 58 4.2.3 | Does Team Risk Aversion Impact The Selection of creative ideas during concept selection? ...... 59 4.2.4 | Do Student personality traits predict teams’ propensity for creative concept selection? ...... 61 4.3 | Chapter Summary and Discussion ...... 63

5 THE EFFECTS OF GENDER, IDEA GOODNESS AND OWNERSHIP BIAS IN CONCEPT SELECTION ...... 66

5.1 | Methodology ...... 67 5.1.1 | Participants ...... 67 5.1.2 | Procedure ...... 68 5.1.3 | Metrics ...... 69 5.2 | Data Analysis and Results ...... 71 5.2.1 | The Relationship Between Idea Ownership and the Selection of Ideas ...... 72 5.2.2 | The Impact of Gender on Ownership Bias during Concept Selection ...... 73 5.2.3 | The Impact of Idea Goodness on Ownership Bias during Concept Selection ...... 74 5.2.4 | Do Higher Order Interactions Affect Ownership Bias? ...... 75 5.3 | Chapter Summary and Discussion ...... 76

v 6 DEVELOPING A SCALE FOR ASSESSING PREFERENCES FOR CREATIVE CONCEPT SELECTION IN ENGINEERING DESIGN ...... 80

7 INVESTIGATING THE IMPACT OF PREFERENCES FOR CREATIVE CONCEPT SELECTION ON DESIGNER BEHAVIOR ...... 97

7.1 | METHODOLOGY ...... 98 7.2 | Participants ...... 98 7.3 | Procedure ...... 98 7.4 | Metrics ...... 101 7.5 | Data Analysis and Results ...... 102 7.5.1 | Are High quality or highly novel Ideas Filtered Out During the Concept Selection Process? ...... 102 7.5.2 | are PCS Factor Scores Related to the Creativity of Generated Ideas? .... 104 7.5.3 | do PCS Factor Scores Predict the Creativity of the Selected Ideas Above and Beyond Personality Traits? ...... 107 7.6 | Chapter Summary and Discussion ...... 110

8 CONCLUSIONS AND CONTRIBUTIONS ...... 112

8.1 | Understanding the Considerations used to Select Concepts During Informal Concept Selection ...... 113 8.1.1 | Experiment Overview ...... 113 8.1.2 | Review of Major Findings ...... 114 8.2 | Identify the Cognitive Biases and Individual Attributes that Influence Decision- Making and Creativity During Concept Selection ...... 114 8.2.1 | Experiment Overview ...... 115

vi 8.2.2 | Review of Major Findings ...... 115 8.3 | Develop a Framework for Measuring Creative Concept Selection in Engineering Design Education ...... 116 8.3.1 | Experiment Overview ...... 117 8.3.2 | Review of Major Findings ...... 117 8.4 | Implications ...... 118 8.4.1 | Implications for Concept Selection Methods in Industry and Education ... 118 8.4.2 | Implications for Cognitive Biases and Individual Attributes in Concept Selection ...... 120 8.4.3 | Implications for Creative Concept Selection in Engineering Design ...... 122 8.5 | Limitations and Future Directions ...... 123 REFERENCES ...... 126

APPENDIX A: INSTRUCTIONS FOR DESIGN TASKS ...... 140

APPENDIX B: DESIGN RATING SURVEYS (DRS) ...... 146

APPENDIX C: ONLINE SURVEY FOR RISK ATTITUDES AND PERSONALITY TRAITS ...... 156

APPENDIX D: PSYCHOMETRIC SCALE ITEMS ...... 163

vii LIST OF FIGURES

Figure 1: Summary of the areas of research inquiry of this dissertation and the primary contribution of this dissertation...... 8

Figure 2: Key convergent phases for arriving at a final solution in the engineering design process. .... 9

Figure 3: Summary of key individual attributes that can influence creative concept selection and decision-making explored in this dissertation...... 17

Figure 4: Example concepts sketched by participant T08LE...... 27

Figure 5: Example individual concept assessment sheet by participant O26TA...... 28

Figure 6: The sorting of team-generated concepts into the ‘Consider’ category and ‘Do Not Consider’ category by Team 5...... 30

Figure 7: Discussion topics, their total frequency of occurrence, and the number of times the topic led to the selection or rejection of a concept. Not all discussions led to the selection or rejection of a concept, resulting in frequency counts for selection or rejection that do not equal the total frequency of the topic...... 34

Figure 8: Example concept generated by a participant in Team 1 that was considered difficult to maintain and ultimately rejected by the team...... 35

Figure 9: Preliminary analysis conducted in order to investigate the relationship between the confounding factors of education level and the Big 5 Factors of Personality on the frequency of discussion topics...... 40

Figure 10: Team PN scores and the frequency of the ‘Inspires New Ideas’ (left) and ‘Idea Decomposition’ (right) discussion topics...... 41

Figure 11: Team PN scores and the word count of the ‘Compare to Existing Products’ (left) and ‘Idea Decomposition’ (right) discussion topics...... 42

Figure 12: Research questions of Study 2 aimed at extending the findings of Study 1 by directly investigating the impact of Big 5 Factors of Personality on Propensity for Novel/Quality Concept Selection...... 47

Figure 13: Example concepts sketched by participant N03AX...... 50

Figure 14: Quality scores assessed using the 4-point scale...... 51

Figure 15: Example financial risk behavior question from Weber, E.U., et al. (2002)...... 54

Figure 16: Significant negative relationship between team propensity for novel concept selection, PN, and average team ambiguity aversion scores...... 60

Figure 17: The relationship between team propensity for novel concept selection and team agreeableness levels (left) and team conscientiousness levels (right)...... 62

Figure 18: Example concepts sketched by Participant N02ER to address the Greenhouse Grid Design Task...... 69

viii Figure 19: Percentage of total ideas selected for male and female participants, categorized by idea ownership...... 73

Figure 20: Percentage of total ideas selected for ideas that have low Goodness scores (<0.5), and high Goodness scores (>0.5), categorized by idea ownership...... 75

Figure 21: Example survey question in the pcs for assessing creative confidence...... 81

Figure 22: Factor structure of the 4-factor model with added error correlations...... 91

Figure 23: Experiment timeline depicting 2 distinct phases, where participants complete online surveys 1 week prior to the design activities...... 99

Figure 24: The relationship between the quality of ideas selected and the quality of ideas not selected during the concept selection activity...... 104

Figure 25: Hierarchical regression analysis schematic for testing the relationship between the factor scores and dependent variables above and beyond covariates and personality traits. ... 105

ix LIST OF TABLES

Table 1: Correlations between frequency and word count of discussion topics identified in this study...... 39

Table 2: Summary of the first multivariate regression analysis with discussion topic frequencies as the dependent variables. Bolded rows indicate significant results...... 41

Table 3: Summary of the second multivariate regression analysis with discussion topic word counts as the dependent variables. Bolded rows indicate significant results...... 42

Table 4: Examples of ideas in the ‘consider’ and ‘do not consider’ categories...... 56

Table 5: Summary of multivariate linear regression analyses between team PN and PQ scores and risk measures...... 59

Table 6: Summary of multivariate linear regression analyses between team PN and PQ scores and personality traits...... 62

Table 7: Examples of ideas for all three design tasks that scored high and low on Goodness...... 71

Table 8: Descriptive statistics of metrics used in this chapter...... 72

Table 9: Summary of results for the logistic regression analysis including all second-order and third- order interaction effects...... 75

Table 10: Three themes of concepts that have been identified by prior work to affect preferences for creativity...... 82

Table 13: Summary of the hierarchical regression analysis results with Task-Related Novelty, as the dependent variable. Bolded rows indicate significant results...... 106

Table 14: Summary of the hierarchical regression analysis results with Task-Related Quality, as the dependent variable...... 107

Table 15: Summary of the hierarchical regression analysis results with the Novelty of Selected Ideas, as the dependent variable. Bolded rows indicate significant results...... 108

Table 16: Summary of the hierarchical regression analysis results with the Quality of Selected Ideas, as the dependent variable. Bolded rows indicate significant results...... 109

x DISSERTATION PUBLICATIONS

Journal Publications

Toh, C., Strohmetz, A., and Miller, S., 2016, "The effects of gender and idea goodness on ownership bias in engineering design," Journal of Mechanical Design, In Press, 2016 (Special Issue on Design Theory and Methodology).

Toh, C. A., and Miller, S. R., 2015, "How Engineering Teams Select Design Concepts: A View Through the Lens of Creativity," Design Studies, 38(1), pp. 111-138. Nominated for best paper 2015.

Toh, C. A., and Miller, S. R., 2016, "Creativity in Design Teams: The Influence of Personality Traits and risk Attitudes on Creative Concept Selection," Research in Engineering Design, 27(1), pp. 73-89.

Conference Publications

Toh, C., and Miller, S., 2016, "The Preferences for Creativity Scale (PCS): Identifying the underlying constructs of creative concept selection," Proc. Design Engineering Technical Conferences, August 21-24, Charlotte, NC, To Appear.

Toh, C., and Miller, S., 2014, "The role of individual risk attitudes on the selection of creative concepts in engineering design," Proc. ASME DETC, August 17-20, Buffalo, NY, Paper No. DETC2014-35106.

Toh, C., Patel, A., Strohmetz, A., and Miller, S., 2015, "My idea is best! Ownership bias and its influence in engineering concept selection," Proc. Design Engineering Technical Conferences, August 2-5, Boston, MA.

Toh, C., Miele, L., and Miller, S., 2015, "Which one should I pick? Concept selection in engineering design industry," Proc. Design Engineering Technical Conferences, August 2-5, 2015, Boston, MA.

ACKNOWLEDGEMENTS

This dissertation is a culmination of many years of work at Penn State, and would not be possible without the help many key players that have touched my life in one way or another. To my research advisor, mentor, role model, and voice in my head, Dr. Scarlett Miller: Thank you for your invaluable guidance and mentorship. You have shown me what an amazing career and life in academia can look like, and I am doubtful that I would have achieved as much as I did without the opportunities you have thrown my way and your never-wavering optimism. The countless hours you have spent working one-on-one with me, the grueling presentation critique sessions, and the constantly changing writing feedback have strengthened my patience and shaped me into the researcher, teacher, and thinker that I am today. I have never worked with anyone with as much grit, compassion, and hunger for excellence, and I can only hope that I will be as excellent an advisor to your academic descendants. I would also like to thank my Britelab family who has seen me through the most challenging stages of graduate school. Classes and long days in the lab would not be nearly as entertaining without my work wife, Elizabeth Starkey, and the other hard-working members of the lab who provide mutual support and guidance. To the undergraduate members of the Britelab: thank you for your hard work in conducting the research despite your massive course loads and extra-curricular activities- this research would not be possible without your contributions. I would like to extend my gratitude toward my wife, Katherine Toh, for your tireless support and pep-talks even through the darkest days. When I am close to despair, you pick me up and have enough confidence and optimism for the both of us. In the wise words on Gandalf the Grey, you showed me that hardship in life is not for me to decide, but “all we have to decide is what to do with the time that is given us.” Thank you to my parents who patiently waited for me to obtain my many degrees, and Susan and Tim Fletcher, who bore witness to my holiday research catch-up sessions. I am also thankful to the best companions one could wish for (Lianna Newman, Brant Rosenberger, Dawn Rosenbaum, Ian Boswell, Art and Steph Shipkowski) that brighten my world and taught me to have a sense of humor toward life. To the Dungeons and Dragons crew: you inspired me to continue adventuring on the campaign towards my PhD. I am also thankful to the members of my doctoral committee, Dr. Timothy Simpson, Dr. Gül Kremer, and Dr. Sam Hunter. Your guidance and experience in this research have strengthened the focus and rigor of my dissertation. I would also like to thank the National Science Foundation for their support of this work as this material is based upon work supported by the National Science Foundation under Grant No. 1351493.

xii

CHAPTER 1

INTRODUCTION

“Engineering problems are under-defined, there are many solutions, good, bad and indifferent. The art is to arrive at a good solution. This is a creative activity, involving imagination, intuition, and deliberate choice” - Sir Ove Arup (Arup, S., n.d.)., Architectural Engineer

The landscape of design and engineering are constantly changing and evolving, and in order to remain competitive in today’s global economy, companies are required to develop innovative solutions and products. The focus on improving the effectiveness of the design process has led to renewed attention on enhancing engineering education in the form of new nationwide initiatives and calls for action in order to improve engineering education and produce engineers who are capable of addressing important challenges in society (Atkins, R. & Flores, N., 2015), (The White House, 2003). As part of this new surge of interest, research has focused on studying and improving creativity training in engineering education. Since creative solutions contribute the most value to the design process (Fuge, M., et al., 2013), design educators are recognizing that students graduating engineering programs need to be trained for creativity prior to entering the workforce (Martin, D.J., et al., 2003; Nicholl, B. & McLellan, R., 2008; Siu, K.W.M., 2009). While a large focus of creativity training in engineering education has been on helping students develop creative ideas, researchers have argued that the “availability of creative ideas is a necessary but insufficient condition for innovation” (p. 48) (Rietzchel, E.F., et al., 2006). This is because creative ideas are often filtered out during the design process because of people’s inadvertent bias against creativity (Rietzschel, E., et al., 2010). The crucial role that concept selection plays in determining the flow of creative ideas through the design process results in

concept selection acting as the “gatekeeper of creativity” in design. Because of the importance of concept selection, a significant quantity of research has focused on studying and increasing the effectiveness of the concept selection process through the development of formalized concept selection methods which are often taught in engineering education (see for example (Ayag, Z. & Ozdemir, R.G., 2009; Hambali, A., et al., 2009; Jacobs, J.F., et al., 2014; Okudan, G.E. & Tauhid, S., 2008). These methods allow for objective evaluation of concepts during concept selection, and provide a quantitative basis for decision-making during this process (Frey, D., et al., 2009; Frey, D.D., et al., 2010; Hazelrigg, G.A., 2010). While this research provides systematic and objective methods for selecting concepts in design, research in engineering industry has shown that companies often lack a coherent or formal process for selecting ideas (Barczak, G., et al., 2009; López-Mesa, B. & Bylund, N., 2011). Instead, the early phases of concept evaluation typically involve a screening process where the ideas generated in the early phases of design are narrowed down to a few key concepts through informal team discussions (Onarheim, B. & Christensen, B.T., 2012). While engineering educators are instructing students on formal concept selection methods, engineering students are graduating programs without knowledge or expertise in informal concept selection methods being practiced in industry. In addition to our lack of knowledge about the informal selection process itself, other confounding factors such cognitive biases associated with human decision-making (De Martino, B., et al., 2006) can negatively influence informal concept selection. For example, factors such as preferences for visually complex designs (Onarheim, B. & Christensen, B.T., 2012), development time (Kruglanski, A.W. & Webster, D.M., 1996), organizational culture (Amabile, T., 1996), designer personality traits (Kichuk, S. & Wiesner, W., 1998) and ownership bias (Onarheim, B. & Christensen, B.T., 2012) can influence decision making during informal concept selection. In addition to these biases, one of the most salient factors in creative concept selection in engineering design is people’s bias against creativity because while practical ideas are generally considered valuable, individuals tend to be more uncertain about whether a novel idea is practical, error-free, or useful. However, these factors are largely ignored in the engineering design curriculum, leaving it unclear how design outcomes are being influenced by decision-making biases, or how to best prepare engineering students to overcome these biases. One of the challenges faced by research that aims to study creative concept selection in design is the lack of a standard tool or method for assessing creativity during the concept selection stage. While there is no specific method to measure one’s preference for creative ideas, there has

been substantial research on developing tools that measure different facets of creativity (see for example (Guilford, J.P., 1950; Mednick, S.A., 1962; Sternberg, R.J., et al., 1997; Torrance, E.P., 1999; Wallach, M.A. & Kogan, N., 1965)). Although these scales provide a rich and varied foundation for measuring creativity, the vast majority of creativity assessment scales do not measure designers’ preferences or attitudes toward creativity. Instead, they focus on assessing individual creative potential, or the degree to which an individual can generate creative ideas. In addition, prior creativity assessment instruments were not developed in the context of engineering design education, leaving it unclear if the results generalize to design students. This is problematic since without a scale for measuring engineering students’ preferences for creativity, the individual factors that impact whether creative ideas are moved forward or filtered from the engineering design process cannot be investigated.

1.1 | RESEARCH OBJECTIVES AND SIGNIFICANCE

In order to address these research challenges, the main purpose of this dissertation was to contribute to knowledge regarding the factors that impact the selection of creative ideas during the concept selection process in engineering education. Specifically, this dissertation aims to address 3 research objectives:

Objective 1: Understand the considerations used to select concepts during the informal concept selection decision-making process in engineering student teams. This objective was developed to add to knowledge about the informal selection process in design in order to allow for improved design training in engineering education aimed at graduating engineers who are prepared and knowledgeable about informal concept selection techniques currently practiced in design industry. The sub-objectives of Objective 1 are: 1a (Chapter 3): Identify the types of factors discussed when student design teams select or reject ideas during the concept selection process 1b (Chapter 3): Identify the types of factors discussed by student design teams who select more creative ideas during this process.

Objective 2: Identify the cognitive biases and individual attributes that influence decision-making and creativity during the informal selection process. This objective examines the various factors that play a role in concept selection and influence the creativity of ideas selected during this stage. The sub-objectives of Objective 2 are: 2a (Chapter 4): Understand if the creativity of an idea has an impact on its likelihood of being selected during concept selection. 2b (Chapter 4): Understand if creative idea generation ability affects team propensity for creative concept selection. 2c (Chapter 4): Understand if team risk-taking attitudes affect team propensity for creative concept selection. 2d (Chapter 4): Understand if team personality traits (specifically agreeableness, conscientiousness, and neuroticism) affect team propensity for creative concept selection. 2e (Chapter 5): Understand if an individual’s ownership of a generated idea affect their likelihood of selecting it to move forward in the design process. 2f (Chapter 5): Understand if the gender of the participant impacts ownership bias. 2g (Chapter 5): Understand if the “Goodness” of the ideas affects ownership bias. 2h (Chapter 5): Understand if higher order interactions of related variables impact ownership bias.

Objective 3: Develop a framework for measuring creative concept selection in engineering design education. This objective focused on developing and testing a scale for measuring preferences for creativity during concept selection and established a theoretical basis for conceptualizing individual traits that constitute creativity during concept selection in engineering design. The sub-objectives of Objective 3 are: 3a (Chapter 6): Identify and test the internal consistency of the underlying factors of preferences for creativity during engineering education concept selection. 3b (Chapter 6): Develop and validate a scale for measuring preferences for creativity during concept selection in engineering education. 3c (Chapter 7): Understand if the PCS factors obtained in Chapter 6 are related to the creativity of generated ideas above and beyond the effects of personality traits. 3d (Chapter 7): Understand if the PCS factors obtained in Chapter 6 are related to the creativity of ideas selected by participants above and beyond the effects of personality traits.

This research was developed to contribute to knowledge about concept selection and creativity in design and provide a foundation for understanding design decision-making and innovation training in engineering education. Since there is limited data on informal concept selection processes and decision-making in design teams, this research allows for improved training in engineering design. In addition, this research is the first of its kind to develop a comprehensive theoretical foundation for conceptualizing creativity during concept selection and provide theoretical and methodological standards for future research in creativity and design.

1.2 | EXPECTED CONTRIBUTIONS

The first contribution of this dissertation is a detailed understanding of the types of considerations being used during informal concept selection, obtained by observing and analyzing engineering student team discussions during informal concept selection activities. Empirical evidence of the link between discussion topics and creative concept selection was explored as part of this dissertation and is expected to provide a foundational basis for studying creative concept selection. The second contribution of this dissertation is a deeper understanding of the cognitive biases that can impact the team decision-making process and the individual attributes that influence creative concept selection in engineering education. Empirical links between attributes such as risk- taking in the financial domain were established with risk-taking in the creative context. The final contribution of this dissertation includes a comprehensive framework for conceptualizing creativity during concept selection, and a scale for assessing preferences for creative concept selection. This contribution allows future work to leverage this tool for studying and improving design creativity instruction in engineering education, and enables future research directions on creativity and decision-making in design.

1.3 | DOCUMENT OUTLINE

In order to address the research objectives of this dissertation, this document presents a total of 8 chapters detailing the methodologies, findings, and implications of this dissertation. These chapters address the dissertations’ sub-objectives, and are organized as follows: Chapter 2 provides a detailed review of the literature on the design process, concept selection, cognitive biases and individual attributes, and opportunities for further investigation that motivate this dissertation. Chapter 3 then outlines an empirical investigation completed to gain an understanding of the informal concept selection process in design (Sub-objectives 1a and 1b). Chapter 4 examines creativity during the concept selection process (Sub-objectives 2a and 2b) and the role of risk attitudes and personality traits on creative concept selection in a design education context (Sub- objective 2c and 2d). Chapter 5 investigates the impact of ownership bias and its covariates on the informal selection process (Sub-objectives 2e, 2f, 2g, and 2h). Chapter 6 outlines the development and validation a psychometric scale for creative concept selection (Sub-objective 3a and 3b), and Chapter 7 investigates the impact of this scale on designer behavior during concept selection activities (Sub-objective 3c and 3d). Finally, Chapter 8 provides a summary of the findings of this dissertation, and highlights the contributions and implications of this work to the design and engineering education communities.

7 CHAPTER 2

LITERATURE REVIEW: THE CONCEPT SELECTION PROCESS IN DESIGN

This dissertation is grounded in prior research conducted in the fields of engineering, psychology and management sciences. This chapter provides a basis for the dissertation by framing the concept selection process within the broader context of design. Next, the context of teams in design and the team decision-making process are reviewed. Finally, prior research on individual biases and preferences are reviewed for their impact on design decision-making. The contribution of this work in the context of the dissertation work is illustrated in Figure 1.

Figure 1: Summary of the areas of research inquiry of this dissertation and the primary contribution of this dissertation.

2.1 | THE DESIGN PROCESS

Making sound decisions in the early stages of design has been recognized by researchers as a crucial factor in successful engineering design (Antonsson, E.K. & Otto, K.N., 1995). Within this concept development phase, several sub-processes occur for converging on a final realization of the design: identifying customer needs, establishing target specifications, concept generation, concept selection, concept testing, and product development (Ulrich, K.T., et al., 2011), see Figure 2. While identifying customer needs and setting target specifications, designers rely on the design goal and target user in order to direct the design process. Once customer needs and specifications have been established, designers then generate concepts that address the design goal. After initial concepts are generated, the design team then narrows down the pool of potential designs and often combines multiple concepts in the concept selection stage. Once a design has been chosen, further testing and refinement is conducted before the product is considered for production.

Figure 2: Key convergent phases for arriving at a final solution in the engineering design process.

Since the design environment is rich with uncertainty and ‘wicked problems’, researchers have focused on developing frameworks for systematically analyzing the early stages of the design process. This is important because concept development happens early in the design process and has the potential to greatly affect the quality of design outcomes (Duffy, A.A., 1993). In the literature, the concept generation and selection stages have received a wealth of attention. During concept generation, designers are expected to produce highly novel and valuable

ideas that open up the solution space. Thus, designers often generate a large quantity of concepts in this initial stage in order to encourage creative idea generation, which in turn, increases the likelihood of improving design outcomes (Dylla, N., 1991). This is important because creativity has been considered to be important in improving design outcomes and innovation in the design process (Li, Y., et al., 2007). From a practical standpoint, creativity can improve design outcomes (Dylla, N., 1991), and thus, enhance the design process. On a broader note, creativity is fundamental to engineering, and is seen as the core of product innovation (Li, Y., et al., 2007). Thus, researchers have developed formalized techniques that seek to increase design creativity in the early stages of design. In particular, methods such as Brainstorming (Osborn, A., 1957), TRIZ (Altshuller, G.S., 1984), SCAMPER (Eberle, B., 1996), and C-Sketch (Kulkarni, C., et al., 2012), that seek to encourage creativity during idea generation have been researched extensively. These techniques rely on structured techniques that open up the solution space during conceptual design and provide inspiration to the designer when generating initial ideas. This approach to encouraging creativity has received a wealth of attention in the engineering design literature, spanning approximately 172 idea generation techniques that seek to improve creativity (Smith, G.J., 1998). On the other hand, concept selection is described as a convergent process that includes both the evaluation, screening, and selection of candidate ideas (Nikander, J.B., et al., 2014). Therefore, any decision-making process that serves to narrow down the solution space from ideas that were generated during ideation is considered to fall under the broad category of concept selection for this dissertation. Specifically, the first stage of the concept selection process occurs directly after concept generation when the design team is tasked with quickly evaluating dozens of concepts and selecting the ideas with most promise to move forward in the design process (Kudrowitz, B.M. & Wallace, D., 2013). Concepts that were generated in previous stages need to be selected and synthesized into a final solution in order to address the design goal (Ulrich, K.T., et al., 2011). Thus, initial concepts are evaluated for their strengths and weaknesses and for their ability to fulfill customer needs. Design feasibility and the fulfillment of technical specifications are emphasized at this stage, since a successful design should not only be novel, but should also be practical and valuable (Ford, C.M. & Gioia, D.A., 2000). Once concepts have been generated and selected for further development, detailed design of the product is then developed, including the technical drawings and system specifications of the project (Ogot, M. & Okudan-Kremer, G.E., 2006). Aspects of previously selected designs are integrated, and a complete working product is fully specified during this stage. Additional testing of the product is then conducted in order to ensure that the product is error-free and meets all

engineering specifications. Production and manufacturing are the final processes that follow the detail design stage, and constitute the product realization component of the design process. The success of this final stage is largely contingent on making sound decisions during the concept development and selection stages of the design process (Hambali, A., et al., 2009; King, A.M. & Sivaloganathan, S., 1999). In fact, researchers have argued that selecting poor concepts can lead to tremendous and irrevocable redesign costs at later stages of the design process (Okudan, G.E. & Tauhid, S., 2008). Even though product realization is the final stage of the design process and integrates the design efforts of the previous stages, decisions made earlier during concept selection have the potential to impact the final design, and researchers have estimated that the concept selection process utilizes approximately 60-80% of design resources (Duffy, A.A., 1993). Therefore, research efforts should focus on concept selection and methods for increasing the effectiveness and creativity during this stage due to its large impact on final design outcomes.

2.2 | SUPPORTING CONCEPT SELECTION IN DESIGN

The importance of decisions made during the concept selection stage of the design process has been acknowledged by researchers and practitioners alike (Okudan, G.E. & Tauhid, S., 2008). This stage is one of the most challenging phases of the design process (Pugh, S., 1996), due to the multiple, and often conflicting, design requirements considered during this stage. Therefore, systematic decision-making methods have been developed that seek to reduce the computational burden placed on designers that inherently have a limited memory capacity to receive, process, and remember information (Baddeley, A., 2003; Miller, G.A., 1956). These concept selection methods essentially assign attribute values to each generated concept and then attempt to compare and contrast the concepts in order to find an ‘optimal’ solution to the design problem (see (Marsh, E.R., et al., 1993; Pahl, G. & Beitz, W., 1984; Pugh, S., 1991) for examples). Technical feasibility is often the most emphasized consideration (Shah, J.J., et al., 2003), but other factors such as effectiveness (Ulrich, K.T., et al., 2011) and idea compatibility (Sivaloganathan, S. & King, A.M., 1999) are also emphasized during this process. While the uniqueness or originality of the design is an important consideration during this process (Yang, M.C., 2009), these formalized design tools often neglect to consider creativity during the selection process (Genco, N., et al., 2012). In fact,

students are often taught to focus on technical rigor and conventional design solutions during engineering design education (Kazerounian, K. & Foley, S., 2007), further reinforcing the focus on technical feasibility during this process. This focus on specification fulfillment and technical feasibility (Ulrich, K.T., et al., 2011) is in sharp contrast with the concept generation stage that encourages divergent and creative thinking. The vast differences between these two phases make it unclear if and how the creativity fostered in the concept generation phase affects the selection of designs in concept selection. Even though creativity is acknowledged as an important part of concept generation, it has not yet been studied for its role in concept selection. More troubling is the finding that individuals have an unconscious preference for conventional design alternatives due to people’s inadvertent bias against creativity (Rietzschel, E.F., et al., 2010). In engineering education where academic achievement is prioritized by students, researchers have argued that students tend to be less creative and innovative when there is a risk of receiving poor grades (Linnerud, B. & Mocko, G., 2013). These findings point to notion that concept selection is the “gatekeeper” of creative ideas in the design process since creative designs that are generated during the early phases of conceptual design are often filtered out during idea selection. The conflicting role of creativity in the concept generation and selection phases suggest that more research is needed to explore the factors that lead to the decreased role of creativity in the later phases of design. Beyond this neglect of creativity during concept selection, there is also a disconnect between the formalized concept selection methods developed and taught in engineering education and the concept selection methods currently practiced in design industry. While formalized concept selection techniques are taught in engineering education to teach students systematic engineering decision-making skills, the rate of adoption of formal methods is still low. Recent research has shown that there is increasing knowledge transfer of design research into design practice (Telenko, C., et al., 2014), but professional designers are reluctant to adopt formalized concept selection techniques, often opting for more informal and subjective procedures (Salonen, M. & Perttula, M., 2005). Researchers have observed that methods developed and introduced by design researchers are adopted very slowly (Gill, H., 1990) or not at all by practicing engineers in industry (Elder, W.R., 1998). This has contributed to industry professionals rarely utilizing formalized concept selection techniques due to a lack of familiarity and knowledge about these methodologies (Toh, C., et al., 2015). Adoption of formalized methods are also low because companies are generally resistant to change and significant effort must be invested in order for changes to occur in industrial practices

(Ford, J.D. & Form, L.W., 2002). However, researchers have found that practicing designers who utilize formal concept selection tools report more satisfaction with these formal methods compared to informal concept selection practices (Salonen, M. & Perttula, M., 2005), indicating that formalized methods hold value in the design process, but the selection process needs to be investigated in more detail to understand how to best develop formal selection methods that are more readily adopted in industry. The large disparity between methods developed in academia and those practiced in industry has been attributed to design research that is said to happen in isolation, without a detailed understanding of actual industrial practices and context (Stempfle, J. & Badke- Schaub, P., 2002). Specifically, researchers have argued that there is a lack of understanding of the industrial demand and application context of concept selection in industry (Birkhofer, H., et al., 2005). This is problematic since design students need to be educated and experienced in the types of practices conducted in design industry in order to be truly prepared to enter the workforce. While little design research exists outside of formal concept selection techniques, researchers have begun to explore the use of informal concept selection techniques in design industry. Since researchers define concept selection as methods that help in the “comparison of different concepts and the selection of one or more concepts for further investigation, testing, or development,” (p. 9) (López-Mesa, B. & Bylund, N., 2011), informal methods can include the screening of a large number of concepts before the use of formalized selection methods, or the informal selection of generated designs to be tested and prototyped without the use of any formalized selection techniques. These research efforts provide more information on how designers in industry make decisions regarding concepts that have been generated by the design team. For example, research by Salonen, M. and Perttula, M. (2005) found that design teams often select concepts based on concept review meetings where design concepts are discussed in a team setting and team consensus is reached by voting on which designs best address the design goal. Other informal methods that are used for decision-making in engineering design include checklists or voting in order to determine which concept the design team is most in favor of (Dym, C.L., et al., 2002). At other times, an expert or influential stakeholder may simply choose the concept based on personal preference and expertise without clear criteria for assessing the generated designs (Ulrich, K.T., et al., 2011). While the informal selection process remains largely unstructured, Busby, J.S. (2001) identified several important factors that influence this informal decision-making process through a series of unstructured interviews with professional designers. Namely, these findings show that design robustness, novelty, production cost, and effectiveness all play key roles in informal concept

selection practices. Other researchers have shown that premature evaluation or convergence to a solution can negatively impact the idea generation process (Bearman, C., et al., 2011). Still, other studies have shown that designers employ a variety of evaluation and problem-solving styles (Nikander, J.B., et al., 2014) that can result in differences in the creativity of final designs (Kruger, C. & Cross, N., 2006).

2.3 | TEAM INTERACTIONS AND PROCESSES IN DESIGN

The study of the collective and collaborative decision-making process should also be integrated in any research that seeks to investigate informal decision-making practices because design is considered an inherently collaborative process (Bucciarelli, L.L., 1988) that involves intricate communication patterns that inadvertently impact the design process (Heath, T., 1993). Furthermore, design is being recognized and taught as a team process in engineering (Dym, C.W., JW; Winner, L, 2003) in part because products developed by teams have been shown to be of higher quality than those produced solely by an individual (Gibbs, G., 1995) and in part because teams foster a wider range of knowledge and expertise which aid in the development of ideas (Dunne, E., 2000). In addition, teamwork has been shown to increase classroom performance (Hsiung, C., 2012) and encourage more creative analysis and design in engineering education (Stone, N.J., et al., 2006). In other words, team decision-making factors are as important, if not more important in determining the direction of collaborative design processes, and thus must be taken into account when studying naturally occurring design practices. However, research on team decision-making in design education is relatively sparse, leaving to question the processes in which engineering student designers collaborate and make decisions during this process. While research in engineering student team communications during collaborative design discussions is limited, a number of studies have qualitatively explored the team decision-making process in design industry. Many studies in design research analyze the design process as it occurs in practice in order to understand the “deeply collaborative, contingent, contextually-specific, and discursive” (Oak, A., 2010, p. 229) practice of design-decision making (Gero, J.S. & Mc Neill, T., 1998; Yang, M.C. & Epstein, D.J., 2005). For example, Christensen, B.T. and Schunn, C.D. (2008) analyzed the conversations of expert engineering designers during product development meetings

and found that design prototypes tended to reduce the mental stimulation needed for innovative thinking. Protocol studies such as those done by Dorst, K. and Nigel, C. (2001) show that some element of ‘surprise’ is necessary for the development of creative ideas by industrial designers. In an effort to understand more about the factors that can influence design team communication, Stempfle, J. and Badke-Schaub, P. (2002) found that a lack of common understanding among team members occurred frequently, leading to extensive explanation and knowledge sharing sessions between team members. This line of inquiry has led researchers in this field to identify key patterns of communication such as negotiations among team members (Bond, A.H. & Ricci, R.J., 1992), established communication roles (Sonnenwald, D.H., 1996), building on team members’ thoughts and ideas, (Hargadon, A., 2003) and reacting in real-time to team activities (Buchenau, M. & Fulton Suri, J., 2000). Researchers in engineering have also uncovered empirical links between team measures of Big 5 Personality Traits, communication, and performance in engineering graduate student dyads (Macht, G.A. & Nembhard, D.A., 2015). These studies show that team decision-making processes are an important element of concept selection practices, and research that investigates the concept selection process in design must do so in the team context. However, the research lacks data on how these informal team decision-making processes affect the selection of creative ideas in the design process. This is problematic because we still lack knowledge of the factors that can influence design teams’ perceptions and preferences for creativity, or how to best modify and implement concept selection methods that encourage creativity.

2.4 | THE IMPACT OF INDIVIDUAL BIASES AND ATTRIBUTES ON CONCEPT SELECTION

In addition to team-level factors, the combined impact of individual biases and attributes are important to the study of decision-making and creativity in engineering design concept selection. This is because team composition, or the combined effect of each team members’ personal attributes and preferences, has been show to play a crucial role in determining many aspects of team interactions and performance (Somech, A. & Drach-Zahavy, A., 2011). Researchers have shown that team-level attributes are actually complex combinations of individual-

level attributes (McGrath, J.E., 1998), and the impact of team-level attributes on team performance or creativity can be affected by factors such as compatibility of individual attributes (Moos, R.H. & Speisman, J.C., 1962), diversity of attributes (Belbin, R.M., 1981; Hoffman, L.R. & Maier, N., 1961) or creative confidence (Baer, M., et al., 2007). These factors have the potential to greatly influence the concept selection process that is often complicated by the numerous competing objectives designers must consider during this process (e.g., potential development costs and product quality (Okudan, G.E. & Tauhid, S., 2008)) and the limited capacity of human memory (Baddeley, A., 2003; Miller, G.A., 1956). Furthermore, decision-making during this stage is often subject to biases associated with human decision-making (see for example Amabile, T. (1996); De Martino, B., et al. (2006); Kruglanski, A.W. and Webster, D.M. (1996); Onarheim, B. and Christensen, B.T. (2012); Kichuk, S. and Wiesner, W. (1998); Hammond, J.S., et al. (1998); Roese, N.J. and Olson, J.M. (1994); Ross, M. and Sicoly, F. (1979). The impact of these decision-making biases and personal attributes on decision-making and creativity during the concept selection process must be investigated in order to add to knowledge about creative concept selection in design. A total of 4 key biases and personal attributes that have been identified in prior research are reviewed in detail in the following sub-sections, and a summary of the key individual attributes explored in this dissertation are shown in Figure 3. These biases include Bias Against Creativity, Risk and Ambiguity Aversion, Personality Traits (Big 5 Factors of Personality), and Ownership Bias. For the purposes of this dissertation, a distinction between Risk/Ambiguity Aversion and Personality Traits is made since Risk/Ambiguity Aversion finds its roots in the Behavioral Economics literature on financial risks, whereas Personality Traits are conceptualized using the Big 5 Factors of Personality rooted in the Psychology literature and assess different aspects of personality preferences and attributes.

Figure 3: Summary of key individual attributes that can influence creative concept selection and decision- making explored in this dissertation.

2.4.1 | BIAS AGAINST CREATIVITY

One of the most salient factors in creative concept selection in engineering design is people’s bias against creativity. Specifically, while creativity is often set as an important goal, researchers have found that individuals in scientific institutions, organizations, and industry often select conventional ideas over creative ones (Ford, C.M. & Gioia, D.A., 2000; Staw, B.M., 1995). This preference for conventional design alternatives is often done in an unconscious manner (Dijksterhuis, A., 2004), and numerous research studies have found that people tend to have an inadvertent bias against creativity (Bower, G.H., 1981; Mueller, J.S., et al., 2011; Rietzschel, E., et al., 2010). This is said to occur because while practical ideas are generally considered valuable, individuals tend to be more uncertain about whether a novel idea is practical, error-free, or useful (Amabile, T., 1996). Indeed, as individuals experience more uncertainty in a situation, their perceptions of creativity quickly become negative (Bower, G.H., 1981), since individuals are strongly motivated to avoid uncertainty and failure (Whitson, J. & Galinksy, A., 2008). On a similar note, individuals perceive more risk associated with endorsing novel ideas (Rubenson, D.L. & Runco, M.A., 1995) because of the uncertainty regarding the success and social approval of their decisions (Moscovici, S., 1976).

This bias against creativity plays an important role in engineering education and is among the multitude of factors that can affect creative concept selection in the classroom. Researchers have found that students tend to be less creative and innovative when there is a risk of receiving poor grades (Linnerud, B. & Mocko, G., 2013). This is despite the fact that researchers and educators have long since acknowledged the importance of fostering creative thinking abilities and methods in addition to teaching key engineering concepts (Felder, R.M. & Brent, R., 2004). Indeed, researchers have shown that educators tend to dislike students who exhibit creative behavior, even though creativity is touted as an important element of learning (Westby, E.L. & Dawson, V.L., 1995). In engineering education, researchers have found that students do not feel encouraged by their instructors to be creative or open-minded, and often do not search for multiple solutions to a design problem for fear of failing or receiving poor grades (Kazerounian, K. & Foley, S., 2007).

2.4.2 | RISK AND AMBIGUITY AVERSION

Closely related to the concept of bias against creative ideas, an individual’s attitude towards risk and ambiguity have also been shown to affect an individual’s perception of creativity (Rubenson, D.L. & Runco, M.A., 1995) and their creative abilities (Dewett, T., 2007; El-Murad, J. & West, D.C., 2003). In the literature, risk and ambiguity aversion have been conceptualized as relatively constant attributes that capture an individual’s attitudes toward risk (Brockhaus, R.H., 1980). Both risk and ambiguity are important to study in design since many situations in practice involve a degree of uncertainty (Antonsson, E.K. & Otto, K.N., 1995; Bucciarelli, L.L., 1988; Sarbacker, S.D. & Ishii, K., 1997; Weck, O., et al., 2007), requiring the decision-maker to take risks during decision-making. Uncertainty refers to “both the probability that certain assumptions made during design are incorrect as well as the presence of entirely unknown facts that might have a bearing on the future state of a product or system and its success in the marketplace” (p. 1) (Weck, O., et al., 2007). By extension, risk can be used to describe the extent to which there is uncertainty in outcomes given creative effort (Sitkin, S.B. & Pablo, A.L., 1992), where the decision-maker is required to make decisions with less than perfect information (Sarbacker, S.D. & Ishii, K., 1997). Research on individual attitudes toward risk is important to explore since risk-taking is stated to be an essential element of creativity due to its ability to encourage the individual to push boundaries and explore new territories (Kleiman, P., 2008).

While risk refers to situations where outcomes have a fixed probability of occurring, ambiguity refers to situations where outcomes have an unknown probability of occurring (Moore, E. & Eckel, C., 2003), created by missing information that is relevant and could be known (Fellner, W., 1961; Frisch, D. & Baron, J., 1988). Research on ambiguity aversion during design decision making is important since many realistic situations involve both risk and ambiguity (Heath, C. & Tversky, A., 1991) and recent studies have shown that an individual’s tolerance for ambiguity is linked to creativity in problem solving tasks (Charness, G. & Grieco, D., 2013). Although both risk and ambiguity are important elements of design decision-making, prior measures of individual risk and ambiguity attitudes (e.g., domain specific risk-attitudes (Bossuyt, D.L., et al., 2013; Weber, E.U., et al., 2002), and preference of ambiguity to risk (Charness, G. & Grieco, D., 2013) were not developed for use in the context of creative concept selection. Thus, their relationship with risk-taking in a creative task is largely unknown. In addition, the use of more traditional and familiar risk measures such as utility theory (Boyle, P.A., et al., 2012; Boyle, P.A., et al., 2011; Han, S.D., et al., 2012) or prospect theory (Kahneman, D. & Tversky, A., 1979) that utilize financial lotteries have not been tested for their relationship to risk-taking in creative tasks. The work conducted in this area shows that both risk and ambiguity aversion are important factors that impact creativity, but little research has been conducted regarding the possible effects these factors may have on creative concept selection. The conflicting role of creativity in the concept generation and selection phases suggest that more research is needed to explore the factors that lead to the decreased role of creativity in the later phases of design.

2.4.3 | PERSONALITY TRAITS

While attitudes toward risk and ambiguity can be viewed as an aspect of personality traits, other individual attributes, such as the Big Five Factors of Personality (Five Factor Model) framework (Costa, P. & McCrea, R., 1992), used extensively in the psychometric literature, has also been shown to be strongly linked to creativity (Feist, G.J., 2006). The Five Factor Model states that personality has five dimensions: (1) neuroticism, (2) extraversion, (3) openness to experience, (4) agreeableness, and (5) conscientiousness, all of which have been explored by research in psychology for their impacts on individual creativity. Researchers have shown that these Big Five Factors of Personality can play a role in affecting the selection of creative ideas since the composition of team member personality and disposition is one of the most important factors in

determining team performance (Wilde, D.J., 1997) and creativity (Somech, A. & Drach-Zahavy, A., 2011). Specifically, research conducted in engineering design has begun to explore the impact of these factors on creative concept selection, and show that that individual levels of personality traits influence individuals’ levels of preference for creative ideas during idea assessment (Toh, C.A. & Miller, S.R., 2016). However, research on the impact of personality traits on creative concept selection in design teams is relatively sparse in the design literature. In contrast with research in engineering design, there has been extensive research in psychology exploring key factors that lead to creativity and have identified the Big Five Factors of Personality as important correlates of creative achievement. Specifically, the Extraversion, Openness to Experience, and Agreeableness personality traits are most closely linked to creativity (Batey, M. & Furnham, A., 2006). For example, researchers have found that creative achievement is closely related to high levels of extraversion (Stafford, L., et al., 2010), while other studies have shown that openness to experience is important for creativity and innovation (McCrae, R., 1987; Steel, G., et al., 2012). On the other hand, some studies have found that the agreeableness personality trait is negatively related to creativity (Feist, G.J., 1998) since creative individuals are low on agreeableness and “do not adapt to others, but go their own way” (p. 254) (Hoff, E., et al., 2012). Contrasting findings from other research show that agreeableness plays no role in creativity, but other traits such as humility and honestly are more integral to creativity (Silvia, P.J., et al., 2011). To a lesser extent, the conscientiousness and neuroticism personality traits have been linked to creativity, where conscientious individuals have been shown to be more innovative (Steel, G., et al., 2012), and highly neurotic individuals are less likely to have boosts in creativity due to anxiety (Xu, H. & Brucks, M., 2011). While these factors are important in determining overall group performance, researchers have argued that the composition or combination of team member personality and disposition plays a crucial role in team performance and creativity (Somech, A. & Drach-Zahavy, A., 2011). Studies that investigate team-level personality traits typically examine an aggregate of team-member scores in order to study the combined effect of team-member personality traits on overall group creative performance (Mohammed, S. & Angell, L.C., 2003; Reilly, R.R., et al., 2001). These studies show that teams that are composed of individuals with high extraversion and openness, and low conscientiousness have been shown to generate more creative ideas (Baer, M., et al., 2007). However, it has also been argued that conscientiousness, agreeableness, and neuroticism traits are important for group creativity (Bell, S.T., 2007; Goncalo, J.A., et al., 2010; Goncalo, J.A. & Staw, B.M., 2006; Woodman, R.W., et al., 1993).

2.4.4 | OWNERSHIP BIAS

In addition to individual attributes such as risk attitudes and personality traits, cognitive biases also have a great influence on decision-making during engineering design concept selection. Factors that can influence the selection of ideas and perceptions of creativity during this stage include ownership bias (Nikander, J.B., et al., 2014), the bias to select visually complex concepts (Onarheim, B. & Christensen, B.T., 2012), confirmation bias (Hammond, J.S., et al., 1998), satisficing (Ball, L.J., et al., 2001), and task familiarity (Forster, J., 2009). One particular bias that has been shown to be a prevalent problem in engineering design industry is bias toward individually generated ideas (Cooper, S.Y. & Lucas, W.A., 2006). In a general sense, this bias is referred to as the Preference Effect and is defined as a systematic preference for one’s own ideas compared to ideas generated by others (Nikander, J.B., et al., 2014). This preference is said to occur due to the increased attachment of ideas and artifacts owned by the individual, otherwise known as ownership bias (Onarheim, B. & Christensen, B.T., 2012). This is important since ownership bias has been shown to affect the objectivity of the idea selection process, potentially affecting the outcome of the final design (Cooper, R.G., et al., 2002). Since the study of ownership bias in engineering design is still relatively sparse, research in other parallel fields provide a foundation for studying its effect in design decision-making. The ownership bias phenomenon can be traced back to behavioral economics research that showed that individuals felt more appreciation for solutions that they developed, and a greater sense of loss for artifacts that they owned (Kahneman, D., et al., 1990, 1991). This behavior, titled the Endowment Effect, is said to occur because individuals emphasize the effects of losing an object in their possession more than gaining an object not yet in their possession (Kahneman, D. & Tversky, A., 1979). These feelings of ownership are attributed to personal investment and an increased sense of familiarity with the owned artifacts or ideas (Pierce, J.L., et al., 2003). A factor that contributes to ownership bias is the finding that people’s perception of themselves is generally more favorable compared to that of other people, which in turn increases bias during decision-making (Alicke, M.D., 1985). Indeed, researchers have argued that ownership bias arises from self-bias, or cognitive self- serving bias, where people inflate the value of their own objects as a means of enhancing their own self-images (Barone, M.J., et al., 1997). For example, in a study where participants were given individual cues before a group decision-making discussion session, initially owned information was considered more important compared to pieces of information discovered through the group

discussion (Chernyshenko, O.S., et al., 2003). Other researchers also argue that ownership bias arises since people tend to question the validity of others’ information, but instead trust their own information, or information confirming their own opinions (Van Swol, L.M., 2007; Wittenbaum, G.M., et al., 1999). Research on the Endowment Effect has shown that these preferences can vary with gender (Kitayama, S. & Karasawa, M., 1997; Pelham, B.W., et al., 2001), This has been attributed to earlier research that has shown that males tend to have higher levels of global self-esteem (Kling, K.C., et al., 1999) and more positive perceptions of performance compared to their female counterparts (Beyer, S., 1990). In contrast, females tend to underestimate their abilities (Lenney, E., 1977) and attribute their success to external factors more often when compared to males (Meehan, A.M. & Overton, W.F., 1986). Recent research conducted on engineering and science teams has shown that females tend to evaluate their male and female counterparts more positively in contrast with male evaluators (Joshi, A., 2014), highlighting the gender differences in self-confidence and perspectives of the self. This may be due to the finding that females are viewed as less competent compared to their male counterparts in highly masculine settings such as engineering teams (Carli, L.L., 2010; Ridgeway, C.L. & Smith-Lovin, L., 1999). Furthermore, early research on self-evaluations of performance has shown that females tend to hold low expectations of themselves on masculine tasks, leading to overly-negative evaluations of their performance (Beyer, S., 1990). The combination of the highly masculine engineering team setting with females’ tendency to hold overly-negative perceptions of their performance and attributing their success to external factors can lead female engineering designers to prefer other team members’ ideas and not take credit for their contributions. While the effects of gender on decision-making has been demonstrated (Bornmann, L., et al., 2007; Pearsall, M.J., et al., 2008), little is known about the impact of gender on ownership bias. This is important since the growing presence of women in historically male-dominated fields such as engineering require that all team members recognize and utilize each others’ strengths to solve complex problems (Harrison, D.A. & Klein, K.J., 2007). Therefore, research should investigate these factors in the design setting in order to fully understand the effects of cognitive biases in engineering education. In addition to gender differences in ownership bias, the effect of the “Goodness” of an idea needs to be explored in a design setting. This is important since it is not yet known if ownership bias occurs regardless of idea quality, or if designers are simply selecting “good” ideas that they themselves generated. In concept selection, teams deliberate over the concepts generated during previous stages and evaluate the quality of an idea (Takai, S. & Ishii, K., 2010). This notion of the

goodness of an idea has been used by researchers to capture the multi-faceted and over-arching quality or value of a generated idea, and can include aspects such as technical rigor, idea performance, creativity, detail, or viability of an idea (Amabile, T.M., 1983; Christiaans, H., 2002; Kudrowitz, B.M. & Wallace, D., 2012; Nelson, B.A. & Wilson, J.O., 2009; Shah, J.J., et al., 2003). Good ideas have also been framed in terms of team-level factors, and has been defined as ideas that possess the most conceptual connections among a given pool of ideas generated by a small design team (Goldschmidt, G. & Tatsa, D., 2005). This is due to the fact that good ideas typically make more connections between previous decisions made by a team, which lead to higher rates of selection for these ideas (Goldschmidt, G. & Tatsa, D., 2005). Other studies have suggested that high quality, or good ideas result from high levels of discussion during concept selection, effectively reducing ambiguities and uncertainties during the subsequent evaluation of the idea (Gutierrez, E., 2009). Similarly, design teams perceive ideas to be higher quality if sufficient group discussion prior to evaluation occurred (Van De Van, A. & Delbecq, A.L., 1974).

2.5 | OPPORTUNITIES FOR INVESTIGATION

Prior research conducted in engineering design, psychology, behavioral economics, and cognitive science all contribute to a foundational understanding of decision-making in teams. However, these studies do not explore creativity in concept selection and do not provide knowledge on cognitive biases and individual attributes in design team decision-making. Specifically, while prior studies provide a foundation for investigating concept selection practices, the retrospective (interview) nature of the studies, focus on professional designers, or lack of emphasis on team- based design discussions leave to question what factors of the design are discussed during student team concept selection processes. Furthermore, these studies did not investigate the factors that encourage the selection of creative ideas, leaving to question the role of creativity during informal team concept selection discussions. More detailed research is also needed to understand the possible effects of risk attitudes on creative concept selection in a design context and to link creativity in other domains to creative risk-taking in design. In addition, prior research has highlighted the importance of individual personality traits in influencing team-level creativity, but also show conflicting findings on which

personality traits can impact team creativity. While research conducted in this area investigated the impact of personality traits on a team’s ability to generate creative ideas, these exists little data on how personality traits can affect a team’s ability to recognize and select creative ideas. This is important since design creativity does not end in the ideation stage of the design process but rather is important throughout the entire conceptual design process in order to effectively increase design creativity. Therefore, research that investigates the impact of team-level personality traits on creativity during concept selection will add to our understanding of how to best increase creativity throughout the design process. The previous sections also highlight the importance of ownership bias, gender effects, and the impact of idea “Goodness”, but information regarding these biases during concept selection is still limited. For example, while the effects of gender on decision-making have been demonstrated (Bornmann, L., et al., 2007; Pearsall, M.J., et al., 2008), little data exists on how gender affects ownership bias in concept selection. This is important since it is not yet known if there are gender differences in decision-making biases such as ownership bias, making it difficult to understand how to best implement strategies for reducing the gender gap in engineering fields (Hutchison, M.A., et al., 2006). Prior studies also highlight the important role that the goodness of an idea can play during concept selection, but it is not yet known if idea goodness affects ownership bias in concept selection. This is important to investigate to understand if designers are consistently showing preference for their own ideas despite the lower quality of their ideas, or if designers are simply selecting good ideas that they themselves generated. This dissertation aims to overcome these research gaps by investigating the informal selection process in engineering education and examining the factors that affect creativity and decision-making during concept selection in engineering education. The following chapters include detailed descriptions of the methodologies used to address the research objectives and the findings and implications of this dissertation.

CHAPTER 3

HOW ENGINEERING STUDENT TEAMS SELECT DESIGN CONCEPTS

The previous chapter highlighted that creative ideas developed during the early phases of design are often rapidly filtered out during the concept selection process (Rietzchel, E.F., et al., 2006) with few making it to commercialization. This is due in large part to due to their inadvertent bias against creative ideas (Rietzschel, E., et al., 2010) and the nature of informal concept selection processes (Kijkuit, B. & van der Ende, J., 2007). Specifically, prior research has shown that individuals often select conventional or previously successful options during this process instead of novel ones during the concept selection process (Ford, C.M. & Gioia, D.A., 2000). Because of this, the concept selection process is often seen as the ‘gate keeper’ of creative ideas. While the process of selecting concepts that satisfy design goals has been regarded by researchers as one of the most difficult and elusive challenges of successful engineering design (Pugh, S., 1996), little is known about how concepts are selected during this process or what factors affect the selection of creative concepts. The purpose of the current chapter is to introduce a research study that was developed to examine the concept selection process in student engineering design teams and identify the factors that impact the selection of creative concepts during this process (Objective 1 of this Dissertation). This was accomplished through the collection of data from an idea generation session and audio recordings of team concept-selection processes. Data were analyzed through a combination of qualitative content analysis and statistics. The results of this study add to our understanding of team-based decision-making during concept selection and highlight the need for encouraging creativity throughout the concept selection process.

3.1 | METHODOLOGY

As a first step towards examining the decision-making process and creativity in the informal selection process, the current chapter of this dissertation was developed to address the following sub-objectives: (1a) identify the types of factors discussed when student design teams select or reject ideas during the concept selection process, and (1b) identify the types of factors discussed by student design teams who select more creative ideas during this process.

3.1.1 | PARTICIPANTS

Thirty-seven engineering students (25 males, 12 females) participated in this experiment. Nineteen of the participants were recruited from the first-year introduction to engineering design course (EDSGN100) at Penn State, while the remaining 18 participants were recruited from a third- year mechanical engineering design methodology course (ME340). Participants in each course were in 3 and 4-member design teams that were assigned by the instructors at the start of the course based on prior expertise and knowledge of engineering design (four 4-member teams, seven 3- member teams). This team formation strategy was used to balance the a priori advantage of the teams through questionnaires given at the start of the semester that asked about student proficiencies in 2D and 3D modeling, sketching and the engineering design process.

3.1.2 | BRAINSTORMING ACTIVITY

At the start of the study, participants were given a brief introduction to the purpose and procedure and were asked to complete an informed consent document. Participants then attended a design session where they were asked to develop a device to froth milk. One of the most elusive challenges of design research is selecting a task that is both representative of the design area and appropriate for the research questions being explored (Kremer, G.E., et al., 2011). The design task chosen in the current chapter, and in many studies throughout this dissertation, was selected to represent a typical project in a cornerstone, or first year, engineering design course. In these courses, students are typically directed to redesign small, electro-mechanical consumer products

that are equally familiar, or unfamiliar, to the student designers (Simpson, T. & Thevenot, H., 2007; Simpson, T.W., et al., 2007). This type of task is often selected because of the minimal engineering knowledge students have in these early courses. In order to ensure our participants were equally familiar with the product being explored, our design task went through pilot testing with first-year students prior to deployment. Specifically, relevant background information and the design problem for the current chapter were provided to participants in written form on paper, as seen in Appendix A: Instructions for Design Tasks. The design task involved developing concepts for a new product, and read as follows:

“Your task is to develop concepts for a new, innovative, product that can froth milk in a short amount of time. This product should be able to be used by the consumer with minimal instruction. Focus on developing ideas relating to both the form and function of the product.”

In addition to the written instructions to generate innovative ideas, participants were also verbally reminded that the goal of the design task was to generate innovative early-phase design ideas instead of focusing on the feasibility or detailed design of the product. Once the design problem was read and understood, each participant was provided with individual sheets of papers and given 20 minutes to individually sketch as many concepts as possible for a novel milk frother. They were instructed to sketch only one idea per sheet of paper and write notes on each sketch such that an outsider would be able to understand the concepts upon isolated inspection, see Figure 4. Twenty minutes was selected for the ideation task because prior research has shown that most creative ideas emerge only after about 9 ideas have been generated (Kurdrowitz, B. & Dippo, C., 2013) and creative idea generation tapers off at around 9 to 10 minutes of ideation time (Beaty, R.E. & Silvia, P.J., 2012; Parnes, S.J., 1961).

Figure 4: Example concepts sketched by participant T08LE.

3.1.3 | CONCEPT SELECTION ACTIVITY

After the brainstorming session, participants were asked to individually evaluate the concepts that had been generated by their team during the previous session. In order to accomplish this, each team member was provided with a stack of ideas (anonymous) from one of their team members and was asked to assess all of the concepts using the provided concept assessment sheets into the following categories (see Figure 5 for example):

Consider: Concepts in this category are the concepts that will most likely satisfy the design goals; you want to prototype and test these ideas immediately. It may be the entire design that you want to develop, or only 1 or 2 specific elements of the design that you think are valuable for prototyping or testing.

Do Not Consider: Concepts in this category have little to no likelihood of satisfying the design goals and you find minimal value in these ideas. These designs will not be prototyped or tested in the later stages of design because there are no elements in these concepts that you would consider implementing in future designs.

Figure 5: Example individual concept assessment sheet by participant O26TA.

These two categories were chosen to simulate the rapid filtering of ideas that occurs in the concept selection process in industry (Rietzchel, E.F., et al., 2006). Participants were also asked to provide their % confidence in their decision to either consider or not consider an idea on the concept assessment sheet. Once the participants had completed assessing all of the ideas from their team

member, they were asked to shuffle the ideas (to avoid any ordering bias), and pass the ideas clockwise to the next team member. This process was repeated until all of the team’s design concepts were assessed by all team members, including each team member’s own ideas. Therefore, each member in the 4-person teams assessed a total of 4 design sets, whereas each member in the 3-person teams assessed a total of 3 design sets, corresponding to each member in the design team. This idea assessment was conducted individually. It should be noted that in order to minimize potential bias, participants were not allowed to share their concept assessment sheets during the activity nor were they allowed to discuss their ratings with their team members during the activity. For the purpose of the current chapter, this individual assessment sheet was not analyzed. Once the individual concept assessment was complete, the teams were given instructions for the team concept selection session, see Appendix A: Instructions for Design Task for the instruction sheet. Specifically, the teams were given the following task for this activity:

“…review and assess the concepts that you and your team have generated to address the design goal in a team setting. Once again, the goal of this design problem is to develop concepts for a new, innovative, product that can froth milk in a short amount of time.”

Participants were asked to discuss each concept with their team members and once a team consensus was made, categorize the concepts into the consider and do not consider categories used in the previous individual concept assessment category. The design teams were asked to physically sort the generated concepts into these two categories and rank the ideas in the ‘consider’ category using post-it notes (1 being the best), see Figure 6. The team dialogue that took place during the discussions was audio-recorded using iPads placed at each team’s workstation.

Figure 6: The sorting of team-generated concepts into the ‘Consider’ category and ‘Do Not Consider’ category by Team 5.

3.1.4 | QUANTITATIVE DATA METRICS

Once the procedures were complete, two independent raters were recruited to assess the creativity of the ideas that were generated using a 20-question Design Rating Survey (DRS) that had been developed in previous studies investigating the creativity of generated designs (Toh, C.A. & Miller, S.R., 2014), see Appendix B: Design Rating Surveys (DRS). The questions on the DRS were used to help the raters classify the features each design concept addressed, similar to the feature tree approach used in the previous studies (Toh, C.A. & Miller, S.R., 2014). The raters achieved a Cohen’s Kappa (inter-rater reliability) of 0.88, and any disagreements were settled in a conference between the two raters after all ratings were completed as was done in previous studies investigating creativity (Chrysikou, E.G. & Weisberg, R.W., 2005). The results from these concept evaluations were used to calculate the following metrics:

Idea Novelty, !" : This metric was developed to capture the amount of novelty in each of the generated ideas. Novelty is the “measure of how unusual or unexpected an idea is compared to other ideas” (Shah, J.J., et al., 2003) and was calculated for each generated design using the feature tree approach developed by Shah, J.J., et al. (2003). In order to accomplish this, the novelty of each feature was first calculated. This feature novelty is defined as the

novelty of each feature, i, as it compares to all other features addressed by all the generated designs. The more frequently a feature is addressed, the lower the feature novelty score.

Thus, feature novelty, #$, can then vary from 0 to 1, with 1 indicating that the feature is

very novel compared to other features. The method of computing #$, is shown in Equation 1.

'() # = * (1) $ '

where T is the total number of designs generated by all participants and C is the total

number of designs that addressed feature #$ . The novelty of each design, j, is then

determined by the combined effect of the Feature Novelty, #$, of all the features that the

design addresses. Because !" is computed for all the features addressed by a design, the

novelty per design, !", is computed as an average of feature novelty, as seen in Equation 2. + ! = * (2) " $

where fi is the feature novelty of a feature that was addressed in the design, and , is the number of features addressed by the design.

Propensity Towards Novel Concept Selection, PN: This metric was developed by the authors to quantify each team’s tendency towards selecting (or filtering) creative concepts during the concept selection process. When developing this metric, the following items were considered:

• Teams should receive a high score for selecting a large number of novel ideas from their idea set. • Teams should receive a low score for not selecting novel ideas if they are present in the idea set. • Teams must not be penalized for the lack of highly novel ideas within their idea set as long as they select the most novel ideas in their set.

Once these guidelines were established, the metric was developed as follows: The average novelty of the selected concepts was divided by the average novelty of all ideas generated by the team. This metric is shown in detail in Equation 3.

B /012/31 4501678 5+ 9161:71; :54:1<79 ?CD(>?× )?) 6 -. = = × F (3) /012/31 4501678 5+ 31412/71; :54:1<79 E ?CD >?

where -. is the team’s propensity for creativity during concept selection, k is the number

of ideas selected by the team, l is the total number of ideas generated by the team, Dj is the th novelty score of the j idea, and Cj = 1 if the idea is selected and 0 if the idea is not selected.

In essence, PN measures the proportion of novel idea selection out of the total novelty of the ideas that were developed by the design team. This metric can achieve a value greater than 1 if the average novelty of the selected ideas is higher than the average novelty of all

the generated ideas, indicating a propensity for creative concept selection. PN can also be less than 1, indicating an aversion for creative concept selection. A score of 1 indicates that the team chose a set of ideas that, on average, had the same novelty as the ideas that they generated, indicating no propensity or aversion towards creative concepts during the selection process. In order to classify teams based on their level of creative concept

selection, teams that scored above the mean score (PN = 1.01) were considered to have high

PN, whereas teams that scored below the mean were considered to have low PN.

3.1.5 | QUALITATIVE DATA CODING PROCEDURE

In all, participants generated 251 ideas and selected 91 ideas during concept selection. This resulted in 265 minutes of audio dialogue that was transcribed and coded by two independent coders. The transcripts of the team dialogue were then analyzed using principles of inductive content analysis (Mayring, P., 2004) in NVivo v.10 (QSR, 2012). The limited and fragmented prior knowledge about student team discussion topics during concept selection makes this method useful for analysis in this chapter (Lauri, S. & Kyngas, H., 2005). Following this approach, the team dialogue was analyzed sentence-by-sentence through open coding, and initial categories of discussion topics were created. The two coders identified

instances of discussions (defined as a block of dialogue between the team members on a particular topic) and classified these discussions into either ‘consider’ or ‘do not consider’ based on team decisions. Next, general themes regarding discussion topics were identified, and the number of instances of discussion topics, as well as their word counts were computed. Similar categories were then grouped together to reduce the number of categories (Burnard, P., 1991), in order to sufficiently describe the types of topics student teams discussed during concept selection. The development of these themes and their sub-categories were directed by the content of the team discussions as well as prior research that provide a foundation for the types of factors that influence the decision making process in engineering design (e.g., feasibility, robustness, novelty, production cost, effectiveness) (Busby, J.S., 2001; Nikander, J.B., et al., 2014). While other methods of analyzing design team communication such as Linkography (Goldschmidt, G., 2014; Kan, J.W.T. & Gero, J.S., 2008) and Latent Semantic Approach (Dong, A., 2005; Dong, A., et al., 126; Fu, K., et al., 2010) have been developed and applied in the field of engineering design Content Analysis was chosen due to its ability to process large volumes of data with relative ease in a systematic manner (Crowley, B.P. & Delfico, J.F., 1996). The two coders achieved an inter-rater agreement of 79.5% for this initial analysis, and any disagreements were settled in a conference between the two raters after all ratings were completed.

3.2 | DATA ANALYSIS AND RESULTS

In order to address sub-objectives 1a and 1b of this dissertation, the data from the generated concepts and the coding of the team discussions was analyzed. The following sections present the detailed results of our analyses in the order of the research questions.

3.2.1 | DISCUSSION TOPICS DURING TEAM CONCEPT SELECTION

The first sub-objective (1a) sought to investigate the factors that impact teams’ decision- making process during the concept selection process. Specifically, the team discussion transcripts were analyzed to uncover general themes behind the selection or rejection of concepts to move on

for further development. In all, 6 main discussion topics and 16 sub-topics were identified; see Figure 7 for the list of these topics and frequency of occurrence. It should be noted that not all discussions led to the selection or rejection of a concept. For example, a participant in Team 4 commented on the technical feasibility of a concept, but the discussion did not lead to the selection or rejection of the idea; “I don’t know if this will work, but I like the idea.” Therefore, the frequency counts for discussions that led to selection or rejection does not necessarily equal the total frequency of occurrence of each discussion topic. The following sections present detailed descriptions and examples of these discussion topics as they occurred during team concept selection discussions.

Discussion Topic 1: Technical Feasibility The discussion topic that was most frequently discussed by the design teams during concept selection was the technical feasibility of the ideas (f = 128), which included discussions about the

ease of execution and effectiveness of a concept in satisfying the design goal. Five sub-topics in this area were also identified including: ability to satisfy design goal (f = 82), maintenance (f = 35), efficiency (f = 13), economics (f = 12), and the manufacturability of the design (f = 2). As can be seen by the frequency of these topics, the majority of the discussions on technical feasibility involved the ideas’ ability to satisfy the design goal. Specifically, the teams often discussed different methods of frothing milk and the ability of each method to froth milk quickly and easily. In other words, teams were focused on whether the generated ideas “worked or not”. For example, a participant in Team 4 commented on a generated design: “That one, I’m not sure how it will work. Like you need another component inside of it to spin and stuff.” Maintenance, or amount of effort and upkeep required of a design, was also frequently discussed in this topic. For example, participants in Team 1 discussed the maintainability of a generated concept (see Figure 8) in detail and eventually decided to reject the concept because it “would be hard to clean”. This focus on the maintenance of the product is consistent with engineering design education that emphasizes meeting customer needs throughout the design process (Ulrich, K.T., et al., 2011).

Figure 8: Example concept generated by a participant in Team 1 that was considered difficult to maintain and ultimately rejected by the team.

Overall, these findings demonstrate that student design teams focus a great deal of their discussions during the concept selection process on the technical feasibility of the generated designs. This finding is supported by prior work that has shown that practical considerations are a vital component of the design decision-making because designs that are impractical or impossible to develop ultimately have no value in the design process (Shah, J.J., et al., 2003). These discussions are also in-line with current educational practices in engineering design that heavily emphasize

design functionality, often relying on well-proven solutions to engineering problems (Kazerounian, K. & Foley, S., 2007).

Discussion Topic 2: Idea Comparison The second most discussed topic during team concept selection involved the comparison of generated ideas with one another (f = 125). These discussions allowed teams to benchmark concepts with previously generated designs and eliminate any redundant ideas. This is important because individuals tend to generate ideas in a ‘train of thought’ manner where successive ideas often share many semantic similarities (Nijstad, B.A., 2002). During these discussions, teams either talked about the Similarity (f = 81) or their Preference (f = 22) for one generated concept over another. Teams often used these discussions to compare the merits and disadvantages of each idea in order to make decisions regarding each generated idea. For example, a participant in Team 2 voiced their preference for one idea over another: “…I like this one better, because when you are using this one you have to have a lot of milk in there...” This process of comparing and contrasting information is common in engineering design since formal concept selection techniques utilize this approach to help designers make effective decisions (Saaty, T.L., 2008). At a more fundamental level, cognitive psychologists have long since recognized the importance of using prior relevant information in order to make judgments (Blumenthal, A.L., 1977). In fact, researchers have shown that the cognitive processes involved in analyzing similarities and making decisions are closely linked (Medin, D.L., et al., 1995), further highlighting the important role that comparisons play in decision-making.

Discussion Topic 3: Similar to Existing Products The third most frequent discussion topic involved comparisons to other similar products that already exist in the market (f = 49). Discussions about existing products served several important roles in facilitating team discussions and were broken down into 2 sub-topics: Explanation (f = 40) and Proof of Concept (f = 9). Design teams often used examples to clarify details and provide further explanation for the generated ideas. Since the design sketches produced by participants were preliminary in nature and occasionally lacked sufficient detail to be clearly understood by the rest of the design team, participants also used existing products as analogies during the team discussion. For example, a participant in Team 1 used an existing product to explain the working principle of their generated concept: “Like two egg beaters. If you’ve ever had an egg beater, it’s just like that.” Other discussions involved using existing products as proof of concepts

or justification of the feasibility of generated ideas. That is, participants would argue that since an existing product uses a specific operating principle, generated ideas that share the same operating principle should be equally successful. These findings show that the use of existing examples is pervasive during team discussions and serves a crucial role in facilitating effective team decision-making. This is supported by prior research that regards the use of existing products as important for benchmarking and is a staple of engineering instruction (Ulrich, K.T., et al., 2011). In addition, researchers have provided evidence for the benefits of using existing examples during the creative process (Herring, S.R., et al., 2009) and have shown that existing solutions to problems encourage analogical thinking and help designers draw insightful similarities between situations (Chan, J., et al., 2011). Other research has shown that ideas that are innovative and distinct from existing products add value to the design process (Yang, M.C., 2009). Thus, these studies show that existing examples serve an important role in stimulating thinking and facilitating decision-making especially during concept selection.

Discussion Topic 4: Inspire New Ideas The fourth topic discussed by participants in this experiment involved discussions that inspired new ideas. During these discussions, team members collaboratively proposed new ideas or elements of an idea amidst the concept selection activity. Since students were explicitly instructed to stop generating ideas and start concept selection, students were not expected to perform idea generation during concept selection. Rather, this discussion topic involved hypothetical conversations among team members regarding changes to the generated ideas that would better address the design goal. These discussions were often motivated by the need to modify an idea in a manner that would make the idea favorable to all team members. This discussion topic was further broken down in 2 sub-topics: Element Modification (f = 24) and Combining Ideas (f = 9). The first sub-topic involved a simple addition or modification of one or multiple elements of a generated design. This occurred mostly because teams favored all but one element of a generated design and concluded that changing that element would make the design successful. For example, a participant in Team 1 suggested a design modification: “Well you know all of yours had wiring going up to the lid but instead you could have it be battery powered.” Design teams also engaged in discussions that led to the combination of two or more ideas that were generated by the team. This process of generating new ideas from existing ideas through the recombination, modification, and adaptation of elements has been recognized as a staple of collaborative design practice (Gerber, E., 2007). In fact, this process has been argued to be crucial to the generation of

truly creative ideas that would not have existed if not for the combination of several designers’ ideas (Hargadon, A., 2003). However, this practice of building on ideas may not be fully encouraged in engineering education since idea generation and concept selection are thought of as disjointed processes that occur one after another, as opposed to in conjunction.

Discussion Topic 5: Creativity The fifth discussion topic, creativity, involved discussions about the uniqueness and originality of a generated design. Discussions about the creativity of the design were divided into either positive elements of the ideas’ Creativeness (f = 23) or the ideas Lack of Creativity (f = 83). Design teams most often engaged in discussions regarding the creative aspects of the generated designs, and used these discussions to break ties between two competing ideas and narrow down the final pool of selected ideas. For example, a participant in Team 2 commented on a generated idea: “This would be a really unique idea and actually applicable.” On other occasions, creative ideas were rejected by teams during the discussions (26% of the time). For example, a participant in Team 10 commented on a generated idea: “It’s fun but not practical. I feel like the milk will get churned or something.” The sub-topic ‘Idea is Not Creative’ involved discussions regarding the lack of creativity in generated designs. Unlike the previous sub-topic that involved discussions either favoring or rejecting creative ideas, this sub-topic typically focused on the disadvantages of unoriginal or redundant ideas. In other words, while design teams may be generally ambivalent about the importance of creativity during concept selection, they unanimously considered ideas that were unoriginal as not useful in addressing the design goal. These results show that the creativity was rarely discussed in team concept selection discussions despite the fact that participants were encouraged to generate creative ideas. In fact, the topic of creativity was the second least discussed topic during team discussions, highlighting the fact that creativity was neglected during the concept selection process. This neglect for creativity is said to occur due to people’s bias against creativity, fueled by the uncertainty and risk associated with novel concepts (Rietzschel, E., et al., 2010). This paradox of creativity in the engineering design process is especially concerning in an educational context since recent research has shown that engineering courses lack instruction and assessment frameworks that encourage creativity in the classroom (Daly, S.R., et al., 2014) often resulting in upperclassmen who are less creative than first-year students (Genco, N., et al., 2012).

Discussion Topic 6: Idea Decomposition The final, and least frequently discussed topic refers to instances when the team decomposes a concept into its sub-elements and considers only one aspect of a design. This discussion topic was divided into 2 sub-topics: Focus on Elements (f = 20), and Disregard Elements (f = 9). Discussions where team members only focus on a single element of a generated concept involve detailed discussions about an aspect of the design that was considered useful. During discussions of the second sub-topic, design teams chose to consider an aspect of the design at the expense of other aspects. That is, design teams selected concepts that only contained a single element worth developing and simply ignored other elements that were not favored by the team. For example, a participant in Team 5 suggested: “Do we want to consider just for the idea of having a pouring mechanism?” The pattern of decomposing concepts into its sub-elements and extracting a single element has been shown to be crucial to effective design thinking and reasoning (Rowe, P.G., 1987). Thus, more focus should be placed on developing instructional strategies that emphasize idea decomposition in order to encourage in-depth discussions and idea flow in a team setting (Ryan, P., 2005). One the discussion topics were identified, correlations between the frequencies and word counts of the discussion topics were explored to gain a better understanding of trends in team discussions during concept selection. Pearson correlation analyses were conducted between all pairs of discussion topics, and the results revealed that there were positive relationships between several discussion topics, see Table 1.

Table 1: Correlations between frequency and word count of discussion topics identified in this study.

Correlated Discussion Topics Pearson Correlation p-value Coefficient, r Idea Comparison and Idea 0.68 0.02 Decomposition Similar to Existing Product and Frequency 0.73 0.01 Idea Decomposition Inspire New Ideas and Similar to 0.62 0.04 Existing Product Idea Comparison and Similar to 0.86 0.001 Existing Product Word Count Similar to Existing Product and 0.72 0.25 Idea Decomposition

3.2.2 | THE IMPACT OF PROPENSITY OF CREATIVE CONCEPT SELECTION ON THE FREQUENCY OF DISCUSSION TOPICS

Once the discussion topics were identified, the relationship between the team propensity for creative concept selection and the frequency and word count of the discussion topics was investigated. Before addressing the sub-objectives, the nature of the data was investigated by exploring the relationship between the novelty of ideas generated and the quality of ideas generated in this study. It was found that there was no significant correlation between these two dimensions of creativity (r = -0.04, p = 0.31). Preliminary analyses were also conducted in order to determine the effects of the confounding factors of education level and Big 5 Factors of Personality scores on the frequency of discussion topics, see Figure 9.

A MANOVA revealed that there was no significant difference in the frequency (Wilk’s λ = 0.17, p = 0.14) and word count (Wilk’s λ = 0.36, p = 0.45) of the 6 discussion topics between the first year and third year students. In addition, the impact of the Big Five Factors of Personality on the frequency and word count of discussion topics was investigated since personality traits have been shown to heavily influence risk-taking, team performance, and creativity (Nicholson, N., et al., 2005; Somech, A. & Drach-Zahavy, A., 2011; Wilde, D.J., 1997; Zuckerman, M. & Kuhlman, D.M., 2000). Regression analyses on the discussion topics revealed that none of the 5 Big Five Factors of Personality significantly predict the frequency of the discussion topics (Extraversion: Wilk’s λ = 0.001, p = 0.06; Agreeableness: Wilk’s λ = 0.001, p = 0.06; Conscientiousness: Wilk’s λ = 0.04, p = 0.34; Neuroticism: Wilk’s λ = 0.01, p = 0.14; Openness: Wilk’s λ = 0.01, p = 0.16). Since the small sample size used in this study limits the use of covariates in the analyses, subsequent analyses explored in the impact of the frequency of discussion topics on the novelty of ideas selected during the design process without taking into account education level and the Big Five Factors of Personality. To address sub-objective 1b, a first multivariate linear regression analysis was conducted with the dependent variables being frequency at which each of the 6 discussion topics occurred

during each team’s discussion, and the independent variable being team propensity for novel concept selection. The results revealed that when taken together, the frequency of occurrence of the 6 discussion topics was significantly impacted by team propensity for novel concept selection, Wilk’s λ = 0.05, F = 13.96, p > 0.01. Specifically, significant positive relationships were found between the frequencies of the ‘Inspire New Ideas’, and ‘Idea Decomposition’ discussion topics and PN, see Table 2 and Figure 10.

Table 2: Summary of the first multivariate regression analysis with discussion topic frequencies as the dependent variables. Bolded rows indicate significant results.

(Discussion Topics) Dependent Variables Frequency of R2 Sig. Occurrence Technical Feasibility 135 0.04 0.57 Compare to Another Generated Idea 103 0.00 0.94 Compare to Existing Products 49 0.21 0.16 Inspire New Ideas 33 0.67 0.00 Creativity 31 0.01 0.83 Idea Decomposition 29 0.49 0.02

Figure 10: Team PN scores and the frequency of the ‘Inspires New Ideas’ (left) and ‘Idea Decomposition’ (right) discussion topics.

A second multivariate regression analysis was conducted with the dependent variable being the word count of each of the 6 discussion topics, and the independent variable being team propensity for creative concept selection. The results revealed that when taken together, the word count of the 6 discussion topics was significantly impacted by team propensity for novel concept

selection, Wilk’s λ = 0.06, F = 10.95, p > 0.02. Specifically, significant positive relationships were found between the word count of the ‘Compare to Existing Products’ and ‘Idea Decomposition’ discussion topics and PN, see Table 3 and Figure 11. It is also interesting to note that while creativity was the second least frequently discussed topic, participants spent the least amount of time on this topic according to the word count frequencies.

Table 3: Summary of the second multivariate regression analysis with discussion topic word counts as the dependent variables. Bolded rows indicate significant results.

(Discussion Topics) Dependent Word R2 Sig. Variables Count Technical Feasibility 3642 0.05 0.51 Compare to Another Generated 2636 0.07 0.44 Idea Compare to Existing Products 1862 0.36 0.05 Inspire New Ideas 1209 0.34 0.06 Creativity 359 0.24 0.12 Idea Decomposition 842 0.60 0.01

Figure 11: Team PN scores and the word count of the ‘Compare to Existing Products’ (left) and ‘Idea Decomposition’ (right) discussion topics.

These results indicate that teams who selected more novel ideas tended to engage in more frequent discussions that Inspired New Ideas, see Figure 11. This finding supports the notion that the co-evolution of the problem and solution space is the “engine of creativity in collaborative design” (Wiltschnig, S., et al., 2013, p. 515). It also adds to our understanding of the factors that contribute to creative concept selection in engineering design. Specifically, student design teams

who spontaneously modify or combine generated ideas ‘on the fly’ during the concept selection process were more successful in selecting novel ideas during this process. This is despite the fact that students are generally taught to generate ideas prior to selecting ideas during formal design training. This result is supported by prior research that has shown that improvising and building on generated ideas is crucial for creativity in design practice (Gerber, E., 2007). This result identifies that encouraging students to not just select concepts, but to evolve their designs during the process can help increase design creativity in the classroom and provide students with further insights into industrial design practices. In addition, it shows that students should be encouraged to really consider the individual aspects of ‘crazy’ ideas in order to identify components that may be useful for further development. The findings of this chapter also show that student design teams that engaged in more frequent and elaborate discussions regarding Idea Decomposition were also found to select more novel ideas during concept selection, see Figure 10 and Figure 11. This result indicates that teams who focused their discussions on single elements of a generated idea and dialogued about the disadvantages and merits of the idea within their teams eventually selected more creative ideas. In addition, these teams also frequently extracted a single favorable element of a generated design to be considered for further development, instead of considering each idea as a complete design that had to be considered at face value. This practice of extracting a single design element and engaging in discussion regarding that element is supported by prior design research on creative idea generation that encourages designers to draw on existing ideas and react in real-time to team generated ideas (Buchenau, M. & Fulton Suri, J., 2000). The fact that student design teams engaged in this creative idea generation method during concept selection further highlights the fact that many of the skills and techniques employed during ideation can be implemented during concept selection in order to increase creativity. Lastly, although there were no significant results for the frequency of occurrence of the ‘Compare to Existing Products’ discussion topic, the word count of this discussion topic was significantly affected by the teams’ propensity for novel concept selection, see Figure 6. This result indicates that teams who dialogued more about comparison to existing products tended to select more creative ideas during concept selection. These teams used existing products as analogies of their generated ideas in order to have detailed discussions about the generated ideas, often benchmarking their ideas against other existing products (Ulrich, K.T., et al., 2011). Although these teams did not necessarily compare their generated ideas to existing products more frequently, the higher word count of these discussions indicate that students were engaging in more lengthy and

detailed discussions and using existing examples to inspire creative thinking through analogical thinking (Chan, J., et al., 2011), improving the creativity of the selected designs.

3.3 | CHAPTER DISCUSSION AND SUMMARY

The first objective of this dissertation was to understand the informal concept selection decision-making process in engineering student teams. In order to accomplish this, the current chapter detailed qualitative and quantitative analysis of team-based discussions by engineering design students. The results of this chapter highlight the following:

• Student design teams most frequently discussed the technical feasibility of generated ideas and often compared generated ideas with one another to make decisions during concept selection • Creativity was mostly neglected during team discussions despite it being emphasized in the earlier stages of the design process, and • Teams that selected more creative ideas tended to compare designs to other existing concepts, were inspired to modify designs during team discussions, and identified useful elements of concepts.

These results have several important implications for engineering design education and research. First, these results show that engineering design students are highly focused on technical feasibility during the concept selection process, as has been emphasized in the engineering curriculum (Kazerounian, K. & Foley, S., 2007). Students often engaged in detailed discussions with team members regarding the relative value and feasibility of generated concepts, citing engineering principles learned from courses and applying key knowledge structures important to rigorous engineering design. However, the findings also highlight the lack of focus on creativity during the concept selection process. While creativity is heavily emphasized in the earlier stages of the design process (Rietzchel, E.F., et al., 2006) and in engineering education (Litzinger, T.A., et al., 2011; Richards, L.G., 1998; Stouffer, W.B., et al., 2004; Sullivan, J.F., et al., 2001), the results from this chapter provide empirical evidence for the neglect of creativity during the concept selection process.

While it is important that students learn to recognize and select viable options during the design process, creativity is an important consideration that can increase the quality of design outcomes (Yang, M.C., 2009) and ultimately help encourage the design of engineering solutions that provide the most value to society. Therefore, it is clear that a re-framing and re-structuring of concept selection practice and instruction in engineering education is necessary if creative ideas are to pass through the concept selection process and ultimately add value to the design process (Rietzchel, E.F., et al., 2006). While the chapter highlights the neglect of creativity during the selection process, future research should be geared at investigating the impact of modifications in educational practices on both the selection of candidate ideas and the final design idea implemented in order to better understand the impact of educational structure on concept selection. In addition to highlighting the neglect of creativity during the concept selection process, the results of this chapter also established an empirical link between the selection of creative concepts and the frequency of discussion topics. Specifically, the results indicate that teams who continue to act on inspiration and generate ideas during the concept selection stage of the design process tend to select more creative ideas. This finding provides evidence for supporting a more streamlined and coherent conceptual design process in engineering design education that truly allows for the co-evolution of problem and solution space (Wiltschnig, S., et al., 2013). This coupled approach to concept generation and selection can not only increase creativity but can also improve the flexibility and effectiveness of the design process. Thus, design instruction and techniques that encourage designers to be inspired through idea generation and selection should be developed and implemented in order to improve the effectiveness of the design process and help encourage creativity. While the current chapter highlighted the neglect of creative ideas during concept selection and identified factors that lead to creative concept selection, there are several important limitations that should be noted. Most important is that this chapter was developed primarily to explore engineering student’s concept selection process in teams in situ through the lens of creativity. Future work should focus on studying design teams in industry to compare the results found in this chapter with design practice. Similarly, larger sample sizes and the investigation of other team- level and individual attributes may reveal a link between creative concept selection and discussions regarding creativity where one was not found in this chapter. Another important point to note is the fact that the current chapter focused on a single design task, and only considered the novelty aspect of creativity, without considering the students’ propensity for quality concept selection. While this chapter provides knowledge of how student designers select novel concepts for a specific design

project, future studies that explore the novelty and feasibility of ideas generated in other design problems throughout the conceptual design process will help validate the results of this chapter. In addition, while the study in this chapter investigated the team conversation in terms of frequency of occurrence and word count of discussion topics, future work that examines more detailed aspects of team discussions, such as the dominance of certain team members during discussions or the impact of personality traits can provide more insights into the team decision-making process in concept selection. Importantly, personality traits, such as the Big 5 Factors of Personality can impact team decision-making and creativity, but the limited sample size used in this study did not allow for analyses to investigate the impact of discussion topics on the novelty of the selected ideas above and beyond these Big 5 Personality trait scores. Therefore, future work, such as the next chapter of this dissertation, should investigate the impact of individual attributes on propensities for creative concept selection during the design process. Finally, while these results showed a link between creative concept selection and the frequencies of these discussion topics, it is not clear if the increased discussion of these topics lead to creative concept selection, or simply if teams with more propensity for creative concept selection naturally engage in more discussions surrounding these topics. Further experimental investigations on this topic will reveal more information regarding the direction of this relationship.

CHAPTER 4

RISK ATTITUDES AND PERSONALITY TRAITS IN CREATIVE CONCEPT SELECTION

The previous chapter highlighted student discussion topics during the informal concept selection process in engineering student teams. Specifically, this research brought to light some underlying biases that may impact the concept selection process such as a focus on technical feasibility rather than idea creativity. In addition, the previous chapter showed that the frequency of discussion topics can impact the novelty of selected ideas during the design process, but did not investigate the moderating role that individual attributes can have on novel concept selection. The current chapter extends this work by directly exploring the impact of individual factors on the selection of creative concepts during the selection processes, see Figure 12.

This research is motivated by prior work that has identified individuals’ bias against creative concepts, which is postulated to occur due to the risk associated with novel concepts (Rietzschel, E., et al., 2010) and the uncertainty behind investing in and endorsing novel ideas (Moscovici, S., 1976; Rubenson, D.L. & Runco, M.A., 1995; Whitson, J. & Galinksy, A., 2008). Consequently, even when creativity is set as an important design goal, researchers have found that individuals in scientific institutions, organizations, and industry often select conventional ideas over creative ones (Ford, C.M. & Gioia, D.A., 2000; Staw, B.M., 1995). This tendency toward conventional ideas occurs because while practical ideas are generally considered valuable,

individuals tend to be more uncertain about whether a novel idea is practical, error-free, or useful (Amabile, T., 1996). This is supported by research that shows that as individuals experience more uncertainty in a situation, their perceptions of creativity quickly become negative (Bower, G.H., 1981), since individuals are strongly motivated to avoid uncertainty and failure (Whitson, J. & Galinksy, A., 2008). While previous work in this area has provided a foundation for understanding perceptions and preferences for creativity, few research studies have explored the impact of risk taking on creative concept selection. The current chapter was developed as a first step towards identifying cognitive biases and individual attributes that influence decision-making and creativity during the informal selection process (Objective 2 of this dissertation) through an exploratory study with 37 engineering students. The results of this chapter highlight the need for a more directed focus on creativity in engineering education in both concept creation and concept selection. The results also add to our understanding of creativity during concept selection and provide guidelines for enhancing the design process.

4.1 | METHODOLOGY

As a first step at addressing Objective 2 of this dissertation, an exploratory investigation was developed to address the following sub-objectives: (2a) understand if the creativity of an idea has an impact on its likelihood of being selected during concept selection, (2b) understand if creative idea generation ability affects team propensity for creative concept selection, (2c) understand if team risk-taking attitudes affect team propensity for creative concept selection, and (2d) understand if team personality traits (specifically agreeableness, conscientiousness, and neuroticism) affect team propensity for creative concept selection. These research questions were built on previous research that found that individual-level risk attitudes can affect creative concept selection and generation in design (Toh, C.A. & Miller, S.R., 2016).

4.1.1 | PARTICIPANTS

Data from the thirty-seven engineering students (25 males, 12 females) who participated in the study described in Chapter 3 of this dissertation were used for the analysis in this chapter.

4.1.2 | PROCEDURE

One-week before the study, participants were introduced to the purpose and procedure of the study and were given an informed consent form to complete. Participants were given brief information regarding the purpose and procedure, but no specific details about the design task, purpose of risk and personality measures, or research hypotheses were disclosed to participants. Therefore, participants were not given any information that could enable them to prepare for the design task in any meaningful way. Once informed consent was obtained, participants were asked to complete an online survey that assessed individual risk aversion and ambiguity aversion using a set of 20 lottery questions (10 each for risk and ambiguity aversion), see the metrics section of this chapter for a description of the questions. The lottery questions were developed and utilized according to established measures used in standard behavioral economics (Boyle, P.A., et al., 2012; Boyle, P.A., et al., 2011; Han, S.D., et al., 2012) in order to capture each individual’s level of risk aversion and ambiguity aversion. In addition, domain-specific risk attitude scores were assessed using Weber, E.U., et al. (2002)’s psychometric scale, and personality measures for each participant were captured using the Short Form for the IPIP-NEO (International Personality Item Pool Representation of the NEO PI-R™) online questionnaire (Johnson, J., 2014). For the purposes of this study, behavioral economics and psychometric measures of risk were delineated from the Big 5 Personality traits in order to capture multi-faceted aspects of personal attributes and preferences in a creative design setting. Participants were assigned unique participant identification code for use in the online surveys and subsequent design tasks in order to maintain participant anonymity. One week after the online surveys were completed, participants attended a design session where they were asked to develop a novel device to froth milk and select candidate concepts. The design task and idea generation procedures used in this chapter were the same methods described in Section 3.1.2 | and in Section 3.1.3 | see Figure 13 for sample sketches.

Figure 13: Example concepts sketched by participant N03AX.

4.1.3 | METRICS

Once the procedures were complete, the survey data and ideas generated by participants were analyzed. Several groups of metrics were developed to answer the research questions, and included Creativity Metrics, Risk and Ambiguity Aversion Metrics, and Personality Trait Metrics.

Creativity Metrics

The designs that were generated were collected and two independent raters were recruited to assess the creativity of all ideas based on the novelty and quality metrics developed by (Shah, J.J., et al., 2003). The results from these concept evaluations were used to calculate the following metrics according to methods described in Section 3.1.4 | of this dissertation: Design Novelty, Participant Task-Related Novelty. In addition to novelty, quality was also considered as a dimension of creativity in this chapter since researchers have often defined creative ideas as ideas that are both novel and useful (Mumford, M.D., 2003, p. 110), and was developed as follows:

Idea Quality, G": Quality is defined as a measure of a concept’s feasibility and how well it meets the design specifications (Shah, J. & Vargas-Hernandez, N., 2003). Similar to Linsey, J.S., et al. (2011), we measured quality on an anchored multi-point scale. However, we included an additional question to the quality scale in order to capture the improvement of the generated concept over the original design. The quality metric was calculated using the raters’ answers to the final 4 questions on the 24-question survey, see Figure 14.

Figure 14: Quality scores assessed using the 4-point scale.

The design quality, G", of each design was then computed using Eqn. 5, where qk is the th answer to the k quality question. qk = 1 when the quality question is answered with a ‘yes’,

and qk = 0 when the quality question is answered with a ‘no’.

I H G = BCD B (4) " J

Propensity Towards Quality Concept Selection, PQ: This metric was developed by the authors to assess each team’s tendency towards selecting or filtering high-quality concepts during concept selection. In order to calculate this metric, first the average quality of the selected concepts is computed. Next, the average quality of all concepts available to choose from is computed. Lastly, the quantity from step 1 is divided by the quantity in step 2. This metric is shown in detail in Equation 6.

B ?CD(K?× )?) 6 -K = × F (5) E ?CD K?

where -K is the team’s propensity for selecting quality ideas during concept selection, k is

the number of ideas selected by the team, l is the number of ideas in their set, and Cj = 1 if the idea is selected and 0 if the idea is not selected.

Task-Related Quality: This metric was developed to capture the level of creativity present in each design team. In order to accomplish this, participant quality metric was first calculated as the average design quality of all the designs each participant generated (Shah, J., et al., 2000; Shah, J.J., et al., 2003), as seen in Equation 7.

K LMNO − QRSMTRU GVMS,TW = ? (6) .

where N is the total number of ideas generated by the participant. Team quality was then computed as the average of the design quality scores for all concepts generated within each design team.

Risk and Ambiguity Aversion Metrics

In addition to measuring the creativity of the ideas generated and selected by each team, the team’s risk attitudes were also measured. Risk attitudes in this study are operationalized as a relatively constant measure of individual preferences and tendencies that relate to decision-making under conditions of risk and uncertainty. Since no measure exists that assesses risk-taking in the context of creative concept selection, and since risk behavior has been shown to vary greatly across situations and domains (Weber, E.U., 2010; Weber, E.U., et al., 2002), it was unclear if, or how well, existing measures of risk could be used to measure risk-taking in a creative domain. Therefore, this work sought to understand the relationship between these exiting approaches for measuring risk taking in a creative task by measuring participants’ risk attitudes according to 2 existing approaches: (1) traditional behavioral economics measures of risk (risk aversion and ambiguity aversion), and (2) psychometric domain-specific measures of risk (financial risk behavior, ethical

risk behavior, and social risk behavior). While 5 domain-specific measures of risk were originally developed using this psychometric approach, the Financial, Ethical, and Social domains of risk were used in this chapter due to their relevance to the social and risk-reward nature of team-based design tasks. On the other hand, the Health/Safety and Recreational domains of risk were not used in this chapter since they do not capture relevant aspects of creative concept selection in a small team setting. Specifically, in order to calculate combined risk attitude scores for each team, the following methods were used:

Risk Aversion: An individual’s risk aversion was measured using the 10 lottery questions (Chronbach’s α = 0.91) from the risk aversion online survey taken from research in standard behavioral economics (Boyle, P.A., et al., 2012; Boyle, P.A., et al., 2011; Han, S.D., et al., 2012). An example question is “Which would you prefer? $15 for sure, or a coin flip in which you get $ [an amount greater than $15] if it is heads, or $0 if it is tails?” Potential gamble gains vary randomly within the interval of $20.00 to $300.00, where monetary increments were determined through a series of pilot tests with engineering students. The team’s combined risk aversion score was calculated as the mean of each team member’s risk aversion score, as is typically done when calculating aggregate attribute scores from individual attribute scores (Mohammed, S. & Angell, L.C., 2003; Reilly, R.R., et al., 2001).

Ambiguity aversion: In addition to risk aversion, ambiguity aversion was also measured due to its significance in the study of decision making since many realistic situations involve both risk and ambiguity (Heath, C. & Tversky, A., 1991). It is important to investigate the role of ambiguity aversion in creative tasks since prior research conducted on ambiguity aversion has shown that an individual’s tolerance for ambiguity is linked to creativity in problem solving tasks (Charness, G. & Grieco, D., 2013), and creative performance (Sternberg, R.J. & Lubart, T.I., 1991; Zenasni, F., et al., 2008). Ambiguity aversion was measured using 10 lottery questions (Chronbach’s α = 0.85) from the ambiguity aversion online survey. The goal of the assessment was to identify the point at which an individual would take the gamble given unknown odds of winning the gamble (i.e., make the ‘uncertain’ choice). An example question is “Which would you prefer? $15 for sure, or $20 if you win the gamble with unknown probability and $0 if you do not?” Ambiguity Aversion was then calculated according to Borghans, L., et al. (2009). Similar to risk

aversion, the team’s combined ambiguity aversion score was calculated as the mean of each team member’s ambiguity aversion score.

Financial Risk Behavior Score: In addition to participants’ financial risk aversion measured using lottery questions, participants’ financial risk behavior was measured from a psychometric perspective using 8 survey questions (Chronbach’s α = 0.70) that assessed each participant’s self-reported likelihood of participating in behaviors that are risky in a financial context on 5-point verbally-anchored Likert scale (Weber, E.U., et al., 2002) through the online survey, see example in Figure 4. While new 7-point scales have been developed for Weber’s psychometric assessment, the use of the 5-point scale strikes a balance between validity and increases in variability that may arise from a larger number of points on a Likert scale (Friedman, H.H. & Amoo, T., 1999).

Figure 15: Example financial risk behavior question from Weber, E.U., et al. (2002).

Ethical Risk Behavior Score: Ethical risk behavior was measured using 8 survey questions (Chronbach’s α = 0.73) that assessed each participant’s self-reported likelihood of participating in ethically risky behaviors on 5-point verbally-anchored Likert scale (Weber, E.U., et al., 2002) through the same online survey (e.g., Forging someone’s signature).

Social Risk Behavior Score: Social risk behavior was measured using 8 survey questions (Chronbach’s α = 0.54) that assessed each participant’s self-reported likelihood of participating in risky social behaviors on 5-point verbally-anchored Likert scale (Weber, E.U., et al., 2002) through the online survey (e.g., Speaking your mind about an unpopular issue at a social occasion).

Big 5 Factors of Personality Trait Metrics

Finally, the Big 5 Factors of Personality were measured using the short Five Factor Model (FFM) online questionnaire (Short Form for the IPIP-NEO (International Personality Item Pool Representation of the NEO PI-R™) (Johnson, J., 2014)). The combined personality trait scores of each team were calculated as follows:

Personality Levels: In order to calculate the aggregate Big 5 Personality trait scores of each design team, the personality traits of each participant was averaged. Each participant received a score (ranging from 0 to 100) on every one of the five personality traits: (1) Extraversion, (2) Agreeableness, (3) Conscientiousness, (4) Neuroticism, and (5) Openness. The team’s aggregate score on each personality trait was then calculated as the average of all the team members’ individual scores, as is typical of team personality research (Mohammed, S. & Angell, L.C., 2003; Reilly, R.R., et al., 2001).

4.2 | DATA ANALYSIS AND RESULTS

During the design tasks, 22 ideas (SD = 6.4) were generated, on average, by each team and 8 ideas (SD = 3.02) were selected, on average, for further development. Examples of ideas that were categorized in the ‘consider’ and the ‘do not consider’ categories are shown in Table 4.

Table 4: Examples of ideas in the ‘consider’ and ‘do not consider’ categories.

Ideas in Mean = 8 ideas ‘Consider’ SD = 3.0 ideas category

Ideas in Mean = 22 ideas ‘Do Not SD = 6.4 ideas Consider’ category

Before addressing the research sub-objectives, a post-hoc power analysis was conducted using the software package, G*Power (Faul, F., et al., 2007). Three predictor variables and a sample size of 11 were used for the statistical power analyses. For moderate to large effect sizes of R2 = 0.70, the statistical power for this chapter was calculated as 0.902. Therefore, it can be concluded that there was adequate power to detect moderate or large effect sizes. Since this chapter has the primary goal of exploring any possible effects that behavioral economics measures of risk, psychometric measures of risk, and personality have on creative concept selection, no interaction effects were explored in the analysis.

In addition, it was also important to conduct some preliminary analysis of our PN and PQ ratio variables in order to identify their appropriateness for analysis. Specifically, in order to ensure a linear relationship between the novelty/ quality of the generated ideas and the novelty/ quality of the selected ideas, two linear regression analyses were conducted. The results revealed that there 2 2 was in fact a significant positive relationship between the novelty (R = 0.53, R adjusted = 0.47, p < 2 2 0.01) and quality variables (R = 0.58, R adjusted = 0.54, p < 0.01). Since these relationships were found to be linear, the PN and PQ ratio variables were found to be appropriate for use in the remainder of our statistical analysis. In addition, to determine the impact of any confounding variables since prior work has demonstrated differences between education levels and creativity in engineering design (Genco,

N., et al., 2012), two ANOVAs were conducted, both using education level as the independent variable. The first ANOVA used team propensity for novel concept selection PN as the dependent variable and the second ANOVA used team propensity for quality concept selection PQ as the dependent variable. The results revealed no significant relationship between education level and

PN, F = 2.10, p > 0.18, and between education level and PQ, F = 0.51, p > 0.49, indicating that education level did not impact the teams’ propensity for selecting novel or quality concepts. Therefore, the data from both classes are analyzed for our analysis. SPSS v.20 was used to analyze the findings. A significance level of 0.05 was used in all analyses, and ordinary least squares methods were used for all regression analyses. The following sections present the detailed results of our analyses in the order of the research sub-objectives.

4.2.1 | DO CREATIVE IDEAS HAVE A HIGHER LIKELIHOOD OF BEING SELECTED DURING CONCEPT SELECTION?

The first sub-objective of this chapter (2a) sought to determine if idea creativity, conceptualized as a combination of novelty and quality, would affect the likelihood of an idea being selected by team members during group concept selection activities. Since the dependent variable of this analysis is discrete (selected or not selected), a multiple logistic regression analysis was conducted on all the generated ideas, with the dependent variable being whether the idea was selected by the team or not. In addition, since creativity is operationalized as the combination of design novelty and quality, the independent variables used in this analysis were idea novelty and idea quality. The results of this analysis revealed that idea novelty and quality did not significantly affect the likelihood of the idea being selected during concept selection, χ2(2) = 3.72, p > 0.16. This result indicates that idea creativity did not significantly affect the selection of ideas during the team concept selection activity. This finding suggests that even if a highly creative design is generated during the early phases of design, it may not be selected during the concept selection process. This result demonstrates that design teams do not show any preference for creative ideas during the selection process, even though creativity is touted as an important element of the design process (Howard, T.J., et al., 2008). Since feasibility is an important element of creativity in this chapter, this result is contrary to prior work that has found that individuals tend to select ideas based on feasibility, rather than originality (Ford, C.M. & Gioia, D.A., 2000; Rietzschel, E., et al., 2010). However, unlike previous studies, the selection activity was conducted in design teams, and

involved typical engineering design problems. Nevertheless, the results of the chapter show that individuals did not show a preference for creative ideas even though creativity is regarded as an important element of successful engineering design. That is, despite the fact that design educators and practitioners recognize the importance of creativity in design, the mere awareness of its importance does not guarantee creative idea generation and selection. Therefore, more focused and directed efforts aimed at highlighting the importance of creativity and encouraging creative activities are needed to increase awareness of creativity throughout the design process.

4.2.2 | DOES CREATIVE IDEA GENERATION ABILITY RELATE TO THE TEAMS’ PROPENSITY FOR CREATIVE CONCEPT SELECTION?

The second research sub-objective of this chapter (2b) sought to determine the effect of team task-related creativity on team propensity for selecting creative ideas during concept selection. In order to address this, a multivariate linear regression analysis was conducted using team propensity for novel concept selection, PN and team propensity for quality concept selection PQ as dependent variables, while team task-related novelty and quality scores were used as independent variables. The multivariate regression analysis revealed no significant relationship between the dependent variables and task-related novelty (Wilk’s λ = 0.86, F = 0.57, p > 0.59), and task-related quality (Wilk’s λ = 0.84, F = 0.65, p > 0.55). These results indicate that task-related creativity is not predictive of the teams’ propensity for selecting creative ideas. In other words, a team’s ability to generate creative ideas has no significant impact on their ability to identify and select creative concepts during the later stages of the design process. This finding suggests that even if a design team generates highly creative ideas, they may not necessarily select these creative ideas during the concept selection process. However, this result is promising because it demonstrates that even if a team does not generate a high number of creative ideas, it does not mean they cannot identify and select the most creative concepts out of their set, and thus contribute significantly to the overall creativity of the design process. Thus, students and practicing engineers who are expected to be creative during the design process should focus on creativity during concept generation and selection in order to truly innovate and break convention. While adoption rates of formalized methods in engineering practice remain relatively low (Birkhofer, H., et al., 2005), the development and study of new methods and techniques for encouraging creativity during the selection phase is essential for increasing design creativity, since

prior research has shown that many existing selection techniques discourage the selection of innovative ideas (Dong, A., et al., 2015). Research efforts aimed at developing and studying these new creativity methods can also help add to our knowledge of the concept selection process in design.

4.2.3 | DOES TEAM RISK AVERSION IMPACT THE SELECTION OF CREATIVE IDEAS DURING CONCEPT SELECTION?

The third research sub-objective of this chapter (2c) sought to determine the effects of team risk attitudes on team propensity for selecting creative concepts. To address this research question, traditional behavioral economics measures of risk (risk aversion and ambiguity aversion) and psychometric domain-specific measures of risk (financial risk, ethical risk, and social risk) were investigated for their effects on the teams’ propensity for creative concept selection. First, a multivariate linear regression was conducted with the independent variables being team risk aversion and ambiguity aversion and the dependent variables being team propensity for novel concept selection, PN and team propensity for quality concept selection PQ scores. This analysis revealed that risk aversion (Wilk’s λ = 0.98, F = 0.08, p > 0.93) and ambiguity aversion (Wilk’s λ

= 0.49, F = 3.71, p > 0.08) could not predict the combination of team PN and PQ scores, see Table 5 for summary. However, team ambiguity aversion scores have a statistically significant effect on

PN scores (B = -0.12, p < 0.05). This result indicates that teams with a lower scores of ambiguity aversion (more tolerance for unknown elements) tended to select more novel concepts, see Figure 16.

Table 5: Summary of multivariate linear regression analyses between team PN and PQ scores and risk measures.

Behavioral Economics Measures Psychometric Domain-Specific Measures Independent Risk Aversion Ambiguity Financial Risk Ethical Risk Social Risk Variables Aversion Behavior Behavior Behavior PN and PQ Wilk’s λ = 0.98 Wilk’s λ = 0.49 Wilk’s λ = 0.91 Wilk’s λ = 0.52 Wilk’s λ = 0.79 combined F = 0.08, p > 0.93 F = 3.71, p > 0.08 F = 0.29, p > 0.76 F = 2.77, p > 0.14 F = 0.82, p > 0.49 PN B = -0.01, p > 0.85 B = -0.12, p < 0.05 B = -0.01, p > 0.77 B = -0.05, p > 0.08 B = 0.03, p > 0.31 PQ B = -0.15, p > 0.72 B = 0.57, p > 0.15 B = -0.08, p > 0.51 B = 0.30, p >0.11 B = 0.11, p > 0.57

Figure 16: Significant negative relationship between team propensity for novel concept selection, PN, and average team ambiguity aversion scores.

A second multivariate linear regression was conducted with the independent variables being team financial, social, and ethical risk behavior scores, and the dependent variables being team propensity for novel concept selection, PN and team propensity for quality concept selection

PQ scores. This analysis revealed that financial risk behavior (Wilk’s λ = 0.91, F = 0.29, p > 0.76), ethical risk behavior (Wilk’s λ = 0.52, F = 2.77, p > 0.14), and social risk behavior (Wilk’s λ =

0.79, F = 0.82, p > 0.49) could not predict team PN and PQ scores, see Table 5 for summary. These results highlight the important role that risk attitudes can play in a design team setting, and show that teams with an overall lower level of ambiguity aversion (higher tolerance for unknown elements) are more likely to select novel concepts. This result is supported by prior research on team creativity that showed that new and original ideas tend to be viewed with skepticism in team settings, likely discouraging the selection of these ideas (Baer, M., et al., 2007). However, teams that are more comfortable with making decisions under uncertainty and who are more willing to select ideas have unknown parameters are more likely to engage in the creative process, negating the general bias against creativity in team settings (Bradshaw, S.D., et al., 1999; Camacho, L.M. & Paulus, P.B., 1995). The fact that no significant relationships were found between risk aversion, financial risk behavior, ethical risk behavior, social risk behavior, and team propensity for novel and quality concept selection in this chapter suggests that perceptions and attitudes toward ambiguity in design dominate in team concept selection tasks, outweighing team

attitudes toward other domains of risk. In addition, the results of this chapter show that ambiguity aversion only plays a role on propensity for selecting creative ideas in the novelty dimension, and not in the quality dimension, suggesting that participants’ perception and preference for novelty may be more affected by team risk attitude factors compared to quality. Nevertheless, since novelty is often considered an essential criteria for innovation and invention (Slaughter, E.S., 1998), and is one of the components of creativity (Shah, J.J., et al., 2003), it is important to study the factors that may affect design teams’ preferences for novel ideas during concept selection.

4.2.4 | DO STUDENT PERSONALITY TRAITS PREDICT TEAMS’ PROPENSITY FOR CREATIVE CONCEPT SELECTION?

The final research sub-objective of this chapter (2d) sought to investigate the impact of the

Big 5 Factors of Personality on the teams’ propensity for selecting novel concepts, PN, and propensity for selecting quality concepts, PQ. In order to understand this relationship, a multiple linear regression analysis was conducted with the dependent variables being team PN and PQ scores, and the independent variables being aggregates of team Big Five Personality scores on all 5 traits *. The multiple linear regression analysis results revealed that aggregates of team Big 5 scores do not significantly predict the combination of both PN and PQ scores, see Table 6 for summary. 2 However, PN scores alone could be significantly predicted by team aggregates of Big 5 scores (R 2 = 0.88, R adjusted = 0.77, p < 0.02). Specifically, higher levels of agreeableness (B = 0.001, p < 0.03) and conscientiousness (B = 0.002, p < 0.04) were found to relate to a higher propensity for novel concept selection in teams, see Figure 17.

*In addition to the mean aggregate of Big 5 Personality scores, the heterogeneity of team Big 5 Personality scores was also used as an independent variable in the analysis. This captures the dispersion or distribution of personality traits within each team, and was computed by taking the standard deviation of all personality scores within the team for each 2 personality trait (Reilly, R.R., et al., 2001). No significant results were found for this analysis (PN: R = 0.34, p = 0.76, 2 PQ: R = 0.28, p = 0.84).

Table 6: Summary of multivariate linear regression analyses between team PN and PQ scores and personality traits.

Extraversion Agreeableness Conscientiousness Neuroticism Openness PN and PQ Wilk’s λ = 0.85 Wilk’s λ = 0.30 Wilk’s λ = 0.30 Wilk’s λ = 0.61 Wilk’s λ = 0.77 combined F= 0.35, p >0.73 F = 4.74, p > 0.08 F= 3.34, p > 0.12 F= 1.29, p > 0.37 F= 0.60, p > 0.59 PN B= 0.000, p >0.42 B= 0.001, p <0.03 B= 0.002, p < 0.04 B= 0.001, p > 0.13 B= 0.000, p > 0.29 PQ B= 0.00, p >0.26 B= -0.003, p >0.54 B= -0.003, p > 0.74 B= 0.004, p > 0.53 B= 0.000, p > 0.95

Figure 17: The relationship between team propensity for novel concept selection and team agreeableness levels (left) and team conscientiousness levels (right).

These results show that the Big 5 Factors of Personality are linked to novelty during concept selection, supporting prior research that has shown that the Big 5 Factors of Personality is related to creative idea generation potential (Stafford, L., et al., 2010). However, the results of this chapter show the Big 5 Factors of Personality only relate to a teams propensity for selecting novel ideas, not their propensity for selecting high-quality ideas. This result suggests that personality may play a larger role in affecting participants’ perception of novelty compared to quality. Specifically, these results show that the agreeableness and conscientiousness traits are positively related to novel concept selection supporting by prior research that shows that teams with high conscientiousness and agreeableness levels are more motivated to achieve goals (Bell, S.T., 2007) and thus, tend to be more creative (Woodman, R.W., et al., 1993). Interestingly, results from other studies that explore these personality traits at the individual level show that agreeableness personality trait is negatively related to creativity (Feist, G.J., 1998), indicating that team-level personality traits may differ from individual-level personality traits at a fundamental level. In fact, researchers have acknowledged that individual attributes interact in complex and dynamic ways in teams, resulting

in team outcomes that are simply more than an aggregation of team-member attributes (McGrath, J.E., 1998). This result suggests that team-based perceptions and preferences for novel ideas is ultimately a function of the composition and heterogeneity of the design team; teams who are composed of many individuals with high creative potential may not necessarily select the most creative ideas and vice versa. In addition, the results of this chapter show that the composition of individual attributes in small design teams can affect the selection of novel ideas in a relatively simple design task, in an engineering education context. Thus, educational strategies that leverage the diverse distribution of individual attributes such as risk attitudes and personality traits should be implemented in order to encourage novel concept selection. In addition, more research efforts are needed to help identify design team configurations that encourage the most creativity throughout the design process.

4.3 | CHAPTER SUMMARY AND DISCUSSION

The current chapter was developed to understand the relationship between creative idea generation ability, personality traits, risk attitudes, and creative concept selection in student design teams. The results highlight that teams that generate highly creative ideas do not necessarily select creative concepts. It was also found that team personality traits and social risk attitudes relate closely to novel concept selection. However, financial risk and ambiguity aversion were not linked to creative concept selection indicating that social risk perceptions dominate team-based concept selection activities. These results serve as an empirical basis for further research on creative concept selection and are used to provide recommendations for design instruction in engineering education. The results of this chapter bear significant implications for research in engineering design education and the instruction of design methods in engineering. First, this chapter provides a better understanding of how concepts are initially screened during the design process, showing that highly creative teams do not necessarily select creative concepts. This chapter also identifies that teaching or encouraging creative concept generation is not sufficient for ensuring the selection of these creative concepts during the later stages of the design process. Therefore, traditional methods of concept selection, such as those that rely on the expected utility framework for selecting concepts

do not take creativity into account and are insufficient for encouraging creativity during the concept selection stage of the design process. This is due to the fact that most concept selection methods do not include creativity as an important aspect of the design while assessing ideas during concept selection. Thus, research is needed to develop and study methods and techniques for encouraging creativity that go beyond the mere expected utility of an idea during concept selection in order to increase overall creativity in the design process. Another important finding of this chapter is that the Big 5 personality traits and risk attitudes are linked to novel concept selection in design. The results provide empirical evidence that team-level Big 5 personality attributes such as agreeableness and conscientiousness affect a design team’s perceptions and preference for the novelty dimension of creativity. While there exists a wealth of prior research that has shown that these Big 5 Factors of personality can greatly affect individual creativity (Batey, M. & Furnham, A., 2006; Furnham, A. & Yazdanpanahi, T., 1995), the effects of these personality traits on team creativity is much less studied (Mumford, M.D., 2012). Some studies have shown that team-level personality traits can influence creative idea generation in teams (Baer, M., et al., 2007; Bell, S.T., 2007; Woodman, R.W., et al., 1993), but few studies have explored team-level personality traits in the context of creative concept selection. The results of this chapter also extends results on the role of team personality and creativity (Baer, M., et al., 2007) that found that high levels of extraversion and openness and low levels of conscientiousness in teams resulted in the generation of highly creative ideas since our results found that high levels of agreeableness and conscientiousness resulted in the selection of more novel ideas. This is supported by prior research that states that the types of cognitive and social factors that influence these two stages of design are fundamentally different and involve different sets of mental processes (Reiter-Palmon, R., 2009). Thus, the formation of teams that have diverse personality traits can help ensure that creativity is encouraged throughout the design process. This notion of beneficial diversity is not novel, as it has been argued by researchers to be crucial in building teams that have high creative performance (Klein, C., et al., 2006). However, this chapter highlights the need for this diversity during the concept selection process. Therefore, efforts to build the ‘perfect’ team composed of individuals with personality traits highly associated with creativity can be seen as a practice in futility since different types of personality traits may be linked with creativity at different stages of the design process. Finally, one of the main purpose of this chapter was to draw a link between team-level risk attitudes and propensities for teams to select creative ideas. The results of this chapter show that social risk attitudes play an important role in the selection of novel ideas in teams, agreeing with

prior research that has shown that creativity is heavily influenced by social factors in a team setting (Woodman, R.W., et al., 1993). In this study, new and original ideas were likely viewed with skepticism in the team, likely discouraging the selection of these ideas. However, teams that are more comfortable with making decisions in ambiguous situations and who are more willing to select ideas with unknown parameters are more likely to engage in the creative process, negating the general bias against creativity in team settings (Bradshaw, S.D., et al., 1999; Camacho, L.M. & Paulus, P.B., 1995). Thus, perceptions and attitudes toward ambiguity appear to dominate in team concept selection tasks, outweighing team attitudes toward other types of risk. The development and adoption of environments and practices that encourage student designers to embrace ambiguity and take risks can allow students to openly and freely discuss ideas and help increase team creativity (Edmonson, A. & Roloff, K., 2009). While the current chapter establishes a link between personality traits, social risk attitudes, and novel concept selection, several important limitations should be noted. Importantly, while this chapter establishes an empirical link between individual attributes and creative concept selection, larger sample sizes and with different types of selection methods such as voting or prototyping may reveal a link between creative concept selection and risk attitudes, such as interaction effects between factors, where one was not found in this chapter. Future work that explores the impact of personality and risk attitude compositions in teams (overall level and spread of traits) using controlled laboratory studies where teams with specific compositions of factors are assigned can also help add to our understanding of how these factors impact creative concept selection. More research is also needed to develop and study risk measures that are appropriate for use in creative contexts, since existing measures of risk may not fully capture the risk-taking behaviors of designers during creative concept selection (low reliability scores for scales). In addition, while these findings provide knowledge of how student designers select concepts for a design project where students were specifically asked to be ‘innovative’, future studies should explore how the concept selection process is impacted in tasks that require varying degrees of innovation.

CHAPTER 5

THE EFFECTS OF GENDER, IDEA GOODNESS AND OWNERSHIP BIAS IN CONCEPT SELECTION

The previous two chapters highlighted student discussion topics during informal concept selection processes in engineering student teams and presented a first step at identifying cognitive biases and individual attributes that influence decision-making and creativity during the informal concept selection process (Objective 1 and 2 of this dissertation). Specifically, Chapter 3 of this dissertation showed that design teams primarily consider technical feasibility during concept selection, and tend to neglect creativity during this process. Next, Chapter 4 investigated the role of risk attitudes and personality traits on concept selection and showed that these individual attributes impact informal decision-making during concept selection (Objective 2). The current chapter extends this work by exploring the impact of ownership bias on the selection of concepts during the design process. This is important because in addition to individuals’ bias against creativity due to the risk associated with novel concepts (Rietzschel, E., et al., 2010), it has been shown that there is also a bias associated with the origination of an idea (e.g. Ownership Bias or Halo Effect). While this bias has been identified in the decision-making process, the impact of these biases on concept selection in the design process and objective decision-making in engineering education is still not clear. Ownership Bias has been shown to be a prevalent problem in engineering design industry, were designers are biased toward individually generated ideas (Cooper, S.Y. & Lucas, W.A., 2006). In a general sense, this bias is referred to as the Preference Effect or Ownership Bias and is defined as a systematic preference for one’s own ideas compared to ideas generated by others (Nikander, J.B., et al., 2014). This preference is said to occur due to the increased attachment of ideas and artifacts owned by the individual, otherwise known as ownership bias (Onarheim, B. & Christensen, B.T., 2012). Importantly, ownership bias has been shown to affect the objectivity of the idea selection process, potentially affecting the outcome of the final design (Cooper, R.G., et

al., 2002). While recent research has shown that ownership bias occurs during concept selection in engineering education, it is still not yet known if other factors can impact the occurrence of ownership bias in engineering. For example, prior research has shown that ownership preferences can vary with gender (Kitayama, S. & Karasawa, M., 1997; Pelham, B.W., et al., 2001), but little data exists on how gender affects the occurrence of ownership bias in design. In addition, the effect of the “Goodness”, or quality, of an idea on ownership bias has yet to be explored leaving to question if designers’ are biased towards their own ideas because they are in fact the best ideas out of the set, or if this ownership bias occurs regardless of how good the idea is. Thus, as a second step towards achieving Objective 2 of this dissertation, the current chapter was developed to identify cognitive biases and individual attributes that influence decision- making and creativity during the informal selection process. This was accomplished through an empirical experiment with 110 engineering design students in the first-year Introduction to Engineering Design (EDSGN100) class at Penn State. The results from this chapter provide empirical evidence of the occurrence of ownership bias in engineering design education and the effects of gender and idea goodness on this bias, and add to our understanding of decision-making biases during concept selection.

5.1 | METHODOLOGY

To complete Objective 2 of this dissertation, an empirical study was developed to address the following sub-objectives: (2e) does an individual’s ownership of a generated idea affect their likelihood of selecting it to move forward in the design process; (2f) does the gender of the participant impacts ownership bias; (2g) does the “Goodness” of the ideas affects ownership bias, and; (2h) do higher order interactions of related variables impact ownership bias.

5.1.1 | PARTICIPANTS

To address these research sub-objectives, a study was conducted with 110 first-year engineering design students (78 males, 32 females) from 4 different sections of EDSGN100.

Similar to in chapter 3, students in each course formed 3 and 4-member design teams that were assigned by the instructors of the course at the start of the semester based on existing knowledge and expertise of engineering design (twenty 4-member teams and ten 3-member teams). The gender balance in these teams was as follows: 18 all male, 5 with 1 female, 7 with 2 females, 1 with 3 females, and 1 all female.

5.1.2 | PROCEDURE

At the start of the design tasks, the purpose and procedure of the experiment was explained using a verbal script. Next, participants were provided with one of the following three design goals via written instructions on individual sheets of paper:

Milk frother (Number of teams = 8): “Your task is to develop concepts for a new, innovative, product that can froth milk in a short amount of time. This product should be able to be used by the consumer with minimal instruction. Focus on developing ideas relating to both the form and function of the product.”

Urinary tract infection (UTI) test strip (Number of teams = 9): "Your task is to develop concepts for a new mechanism to expose test strips to urine samples. This product should be simple, inexpensive, low-waste, and durable and constructed of locally-available materials."

Greenhouse grid (Number of teams = 13): "Your task is to develop concepts for a new tool to determine the levelness the of ground in a 7x7 meter grid for a 6x6 meter greenhouse, and to mark 49 frame post locations which are square. Any one post can be no more than 1 centimeter off and the grid should be completely marked in no more than 10 minutes. The device to measure the levelness should be lightweight and ruggedized for the harsh environment with a budget of $10. The materials are limited to nylon string, wood, and metal bars."

These design tasks were chosen for use in this chapter since it allows for an investigation of ownership across a broader range of design tasks with different levels of complexity and across various domains. The milk frother design task was assigned to all teams in one course section, and

the remaining teams were given free reign to select between the UTI Test Strip and Greenhouse Grid design tasks. Once the design tasks were distributed and read by the participants, they were given the opportunity to ask questions, see Appendix A: Instructions for Design Tasks for instruction sheets. While clarification questions were allowed, no design suggestions or additional information on the design tasks were provided to participants during the question and answer period. Next, the participants were given individual sheets of papers to individually sketch as many concepts as possible to address their assigned design goals in the 20 minutes provided. During this ideation session, the participants were instructed to sketch only one idea per sheet of paper and write notes on each sketch such that an outsider would be able to understand the concepts upon isolated inspection, see Figure 18. No discussion was allowed during this individual brainstorming session.

Figure 18: Example concepts sketched by Participant N02ER to address the Greenhouse Grid Design Task.

During the next class period, two days after the brainstorming session, participants returned for a second design session where they were asked to individually evaluate the concepts generated by their team members using the individual concept evaluation process detailed in Section 3.1.3 |

5.1.3 | METRICS

In order to investigate the impact of the quality of an idea on ownership bias during concept selection, a Goodness metric was developed. The metric of ‘Goodness’ used in this study is defined as the subjective effectiveness or value of a generated idea, and includes aspects such as technical rigor, idea performance, creativity, detail, or the overall viability of an idea (Amabile, T.M., 1983;

Christiaans, H., 2002; Kudrowitz, B.M. & Wallace, D., 2012; Nelson, B.A. & Wilson, J.O., 2009; Shah, J.J., et al., 2003). Idea Goodness was chosen to represent this all-encompassing and subjective measure of idea effectiveness in order to disambiguate from the more common term ‘quality’ used in describing an aspect of idea creativity during ideation activities (Shah, J.J., et al., 2003). While “Good” ideas may include creative ideas, these two terms are conceptually different, and analysis of creativity is beyond the scope of this chapter. In addition, a new “Goodness” metric was developed instead of using existing creativity metrics such as Novelty and Quality since Goodness is an absolute measure of idea effectiveness, whereas Novelty and Quality are relative measures that depend on the dataset of ideas generated in the study. This metric was developed to rate the quality or effectiveness of the idea by assessing the number of team members’ who selected the concept to move forward in the design process. Team consensus or agreement was used as a proxy for the Goodness of the idea in order to capture the team’s collective perception of the idea’s Goodness. This metric was computed for each idea generated using each team members’ decision regarding the idea, excluding the decision of the team member who generated the idea. This was done in order to remove any potential bias related to idea ownership and concept selection. This metric was calculated as shown in Equation 6.

` \ × _ b ]=1 ],Z ],Z Z=1 ` XYYUZRNN[ = b (6)

th th Where Xm,n = 1 if the m team member selected the n idea generated by another member in their team for further consideration, and Xm,n = 0 otherwise. Similarly, Cm,n is the percent confidence of the mth team member on their decision regarding the nth idea. M is the total number of team members in the design team, and N is the total number of ideas generated by the participant. Therefore, a Goodness score below 0.5 indicates that the majority of team members (excluding the team member who generated the idea) selected the idea for further consideration during the concept assessment activity. Examples of ideas with high Goodness scores and low Goodness scores are shown in Table 7.

Table 7: Examples of ideas for all three design tasks that scored high and low on Goodness.

Goodness Milk Frother Greenhouse Grid UTI Test Strip scores

High

Goodness = 0.78 Goodness = 0.96 Goodness = 0.93

Low

Goodness = 0.10 Goodness = 0.25 Goodness = 0.28

Idea Goodness was calculated for each idea assessed in the chapter by each team member resulting in a total sample size of 2,517.

5.2 | DATA ANALYSIS AND RESULTS

Prior to analyzing the research sub-objectives, descriptive statistics were calculated for the metrics, see Table 8. SPSS v.22 was used to analyze the findings with a significance level of 0.05. A post hoc power analysis was conducted using the software package, G*Power (Faul, F., et al., 2007). Three predictor variables and a sample size of 2517 were used for the statistical power analyses. For small to moderate odds ratio = 1.3, the statistical power was calculated as 0.99.

Therefore, it was concluded that there was more than adequate power to detect small or moderate effect sizes.

Table 8: Descriptive statistics of metrics used in this chapter.

Variable Mean SD Number of ideas generated by each participant 5.89 2.51 Number of ideas selected by each participant 3.66 1.12 Proportion of individual’s ideas selected 0.61 0.23 Proportion of team members’ ideas selected 0.63 0.15 Idea Goodness 0.33 0.30

In addition, preliminary analyses were also conducted on the data in order to determine any possible impact of gender and design task on the Goodness metric. Thus, an ANOVA was conducted with the dependent variable being the Goodness of the ideas, and the independent variables being participant gender and design task. The results revealed that gender (F = 2.73, p = 0.10) and design task (F = 1.33, p = 0.25) did not significantly impact Goodness scores of ideas generated in this chapter. Therefore, the combined data from both genders and all design tasks are analyzed for our analysis. Sections 5.2.1 | - 5.2.4 | present the detailed results of our analyses in the order of the sub-objectives.

5.2.1 | THE RELATIONSHIP BETWEEN IDEA OWNERSHIP AND THE SELECTION OF IDEAS

The first research sub-objective of this chapter (2e) addressed the relationship between idea ownership and concept selection. The hypothesis was that participants would be more likely to select their own ideas over the ideas of their team members. Since the dependent variable of this analysis was discrete (selected or not selected), a Logistic Regression was computed with the dependent variable being whether the design was selected or not and the independent variable being the ownership of the idea (generated by the participant or by other members of the team). The results showed that ownership of an idea did not significantly affect the likelihood of the idea being selected during concept selection, χ2(1) = 0.01, p = 0.94 refuting our hypothesis. These results indicate participants, on a whole, did not show any preference for or against their ideas during the concept selection activity.

5.2.2 | THE IMPACT OF GENDER ON OWNERSHIP BIAS DURING CONCEPT SELECTION

The second research sub-objective of this chapter (2f) was developed to investigate the impact of gender on ownership bias during concept selection. It was hypothesized that gender would interact with the relationship between idea ownership and idea selection. In order to answer this question, a logistic regression analysis was conducted with the dependent variable being whether the idea was selected or not, and the independent variables being idea ownership and gender of the participant assessing the idea. The results of the regression analysis revealed that the combined effect of all variables and interaction effects did not predict the likelihood of an idea being selected χ2(3) = 5.28, p = 0.15. In addition, the likelihood of an idea being selected was not significantly affected by idea ownership (Wald Criterion = 3.47, p = 0.06) and gender (Wald Criterion = 3.08, p = 0.08) individually. However, the interaction effect of idea ownership and gender significantly predicted the likelihood of an idea being selected (Wald Criterion = 4.73, p = 0.03) confirming our hypothesis. These results indicate that male participants tended to be biased toward their own ideas by selecting a higher percentage of their own ideas (62.8%), compared to their team members’ ideas (59.6%). On the contrary, the female participants showed a bias against their own ideas by selecting a lower percentage of their own ideas (55.9%) compared to their team members’ ideas (64.3%), see Figure 19.

Figure 19: Percentage of total ideas selected for male and female participants, categorized by idea ownership.

5.2.3 | THE IMPACT OF IDEA GOODNESS ON OWNERSHIP BIAS DURING CONCEPT SELECTION

The third research sub-objective of this chapter (2g) was developed to investigate the impact of the Goodness of the idea on ownership bias. Our hypothesis was that the Goodness of an idea would not affect ownership bias during concept selection since we hypothesized that designers would systematically prefer their own ideas regardless of the quality of the idea. To address this research question, a logistic regression analysis was conducted with the dependent variable being whether the idea was selected or not, and the independent variables being idea ownership and Goodness scores of the idea. The results of the regression analysis revealed that the combined effect of all variables and interaction effects predicted the likelihood of an idea being selected, χ2(3) = 1055.25, p < 0.01. Specifically, the likelihood of an idea being selected was significantly affected by idea ownership (Wald Criterion = 71.7, p < 0.01) and Goodness (Wald Criterion = 428.2, p < 0.01) individually. In addition, the interaction effect of idea ownership and Goodness significantly predicted the likelihood of an idea being selected (Wald Criterion = 137.4, p < 0.00). In order to visualize these results and understand their implications, the idea Goodness metric was broken into 2 distinct categories: (1) ideas with Goodness scores above 0.5, and (2) ideas with Goodness scores below 0.5. These cut-off values were chosen since ideas achieved Goodness scores above 0.5 only if the majority of members on a team (not including the team member who generated the idea) selected an idea for consideration. These results indicate that participants in general, selected a higher percentage of ideas that scored higher than 0.5 on the Goodness metric (92.7%) compared to ideas that scored less than 0.5 on the Goodness metric (46.7%). The results also showed that for ideas that scored less than 0.5 on the Goodness metric, participants were biased towards their own ideas by selecting a higher percentage of their own ideas (53.7%), compared to their team members’ ideas (44.3%), or showing ownership bias. On the other hand, for ideas that scored more than 0.5 on the Goodness metric, participants were biased against their ideas by selecting a lower percentage of their own ideas (76.7%) compared to their team members’ ideas (98.1%), or showing the opposite of ownership bias, see Figure 20.

Figure 20: Percentage of total ideas selected for ideas that have low Goodness scores (<0.5), and high Goodness scores (>0.5), categorized by idea ownership.

5.2.4 | DO HIGHER ORDER INTERACTIONS AFFECT OWNERSHIP BIAS?

The last research sub-objective of this chapter (2h) sought to investigate if higher order interaction effects of ownership, gender, and idea goodness affected ownership bias during concept selection. In order to address this research question, a logistic regression analysis was conducted with the dependent variable being whether an idea was selected or not, and the independent variables being idea ownership, gender, idea goodness, all second order interaction effects (ownership*gender, ownership*goodness, gender*goodness), and the third-order interaction effect (ownership*gender*goodness). The results revealed that all independent variables significantly predicted the likelihood of an idea being selected, χ2(7) = 1062.4, p < 0.01. There were similar patterns of results for individual first-order effects and the interaction of ownership and gender, and the interaction of ownership and goodness as in previous research questions, see Table 9. The second-order interaction effect between participant gender and goodness did not significantly predict the likelihood of an idea being selected (Wald Criterion = 1.48, p = 0.22). In addition, the third-order interaction effect between idea ownership, gender, and idea goodness did not significantly predict the likelihood of an idea being selected (Wald Criterion = 2.50, p < 0.11).

Table 9: Summary of results for the logistic regression analysis including all second-order and third-order interaction effects. Bolded rows indicate significant results.

Independent Variable Wald Criterion p-value

Idea Ownership 3.22 0.07 Gender 1.72 0.19 Goodness 81.43 <0.01 Ownership*Gender 6.16 0.01 Ownership*Goodness 20.38 <0.01 Gender*Goodness 1.48 0.22 Ownership*Gender*Goodness 2.50 0.11

These results show that both male and female students tended to select more ideas that ere considered ‘good’ during concept selection. However, the results of the third-order interaction indicated that while male students displayed ownership bias and female students showed preferences for their team members’ ideas, these gender differences did not depend on the level of goodness of an idea. That is, gender differences in ownership bias persisted regardless of whether the idea was considered good or not.

5.3 | CHAPTER SUMMARY AND DISCUSSION

The second objective of this dissertation was to identify cognitive biases and individual attributes that influence decision-making and creativity during the informal selection process. As a first step towards this objective, the current chapter investigated the role of ownership bias, gender and idea goodness on the concept selection process by studying 110 engineering design students in the first-year Introduction to Engineering Design (EDSGN100) class at Penn State. The results of this chapter highlight the following:

• Male student designers showed ownership bias while selecting concepts, whereas female students showed preference for team members’ ideas during concept selection. • Regardless of participant gender, students exhibited ownership bias towards ideas that had lower goodness scores, whereas students showed a preference for their team members’ ideas over their own for ideas that achieved high goodness scores. • Gender differences for the ownership bias effect were not impacted by the goodness of ideas.

One of the main findings of this chapter was that engineering students exhibited ownership bias during the concept selection process but that the gender of the student impacted its effect. Specifically, this chapter demonstrated that male students tended to exhibit ownership bias by selecting more of their own ideas than their team members. This may be due to the fact that the male students had trouble ‘drowning their own puppies’ during the conceptual design process (Cooper, R.G., et al., 2002; Cooper, S.Y. & Lucas, W.A., 2006). This finding supports prior research conducted in engineering industry which also showed that designers prefer their own ideas over the ideas of others (Onarheim, B. & Christensen, B.T., 2012). The current chapter, however, also refutes this prior work by finding that female students displayed the opposite of ownership bias. That is, female students evaluated their own generated ideas as overly-negative when compared to their team members’ ideas, consistent with prior findings that show that females tend to hold low expectations of themselves on masculine tasks, leading to overly-negative evaluations of their performance (Beyer, S., 1990). In contrast, research has shown that males tend to have higher levels of global self-esteem (Kling, K.C., et al., 1999), causing the gender differences found in this chapter. While the effects of gender on decision-making has been demonstrated (Bornmann, L., et al., 2007; Pearsall, M.J., et al., 2008), this study provides empirical evidence of gender differences in ownership bias in a design setting. These results add to current research on decision- making in the design process and provides further evidence that gender effects play a role in informal concept selection practices in engineering education. Therefore, further research should investigate the role of gender in the engineering classroom to understand how team decision- making is impacted by differences between genders. The results of this chapter have important implications for engineering research and practice since it indicates that evaluations during informal concept selection may not be entirely objective, and are subject to human decision-making biases and gender effects. Therefore, design education that emphasizes these decision-making biases during informal team discussions and design work can help increase students’ awareness of these biases, and work toward reducing their negative impact on concept selection. In addition, systematic and rigorous training on informal concept selection techniques in addition to formal selection methods can help prepare students for design practices in industry and enable objective and effective decision-making during these informal methods. The gender differences in ownership bias found this study also highlight the complexities of team interactions in the engineering classroom, and research and teaching efforts should be conducted that aim to address the overly-negative self-evaluations of female students who do not fit in the dominant culture of engineering. Educational strategies that aim to reduce the

disparity in self-esteem and self-efficacy between male and female engineering students (Hutchison, M.A., et al., 2006) should be developed and implemented in order to address gender differences in ownership bias during concept selection. An empirical link between idea goodness and ownership bias in an engineering education setting was also established in this chapter. Specifically, our results showed that both male and female participants displayed ownership bias for ideas that scored low on goodness, and the opposite effect, for ideas that scored high on goodness. On the other hand, students displayed ownership bias for their own ideas with lower goodness scores. This indicates that there are different thresholds of goodness that a design must reach to be selected by an individual; for a designers’ own ideas, this threshold is much lower, whereas designs generated by other team members must exceed a higher threshold of goodness in order to be selected by designers. In order to explore this finding in further detail, a brief analysis was conducted between the Goodness scores of ideas and the creativity scores, operationalized as novelty and quality (Shah, J.J., et al., 2003), and calculated in the same manner as detailed in Section 3.1.4 | It was found that Goodness did not relate to idea novelty, but was significantly weakly correlated with the quality of an idea (Pearson Correlation Coefficient, r = 0.19, p < 0.001). This indicates that while Goodness may reflect a certain aspect of idea quality, this construct is complex in nature and encompasses a wider range of idea effectiveness than creativity. Although prior research has shown that ownership bias exists in the design process, our results highlight the complex interaction of ownership bias, gender, and goodness in a design setting, indicating that the individual and social psychological processes that underlie cognitive biases do not operate in isolation of one another, but in a multifaceted and inter-dependent manner. The results of this research motivate a call for future work that investigates cognitive biases and other team-level factors for their impact on the concept selection process. Furthermore, since decision-making biases reduce the objectivity of the informal selection process, the adoption and training of formalized methods that are effective and easily implemented in engineering design practice can play a crucial role in increasing the effectiveness of the concept selection process. The results of this chapter provide empirical evidence of ownership bias, its gender effects, and the influence of idea goodness, but there exist several limitations that are important to note. This research sought to isolate and study the effects of ownership bias by using individual ideation and selection activities. However, variations in concept selection practice in industry such as shared ownership of ideas, team discussions, and organizational structure can affect ownership bias in practice, and should be explored in future work. In addition, while the sample size of female

participants was characteristic of engineering design classes and the engineering design profession, controlled laboratory studies with larger sample sizes across various design domains and disciplines should be conducted to investigate ownership bias, gender, and goodness in more detail in order to address potential confounds and uncover the exact relationship between these factors and the underlying psychological constructs that cause them in a design setting. These controlled laboratory studies can also reduce any possible bias in the experimentation of this chapter, such as possible hypothesis awareness or experimenter bias that can influence the effects of ownership bias in a design activity. Lastly, the characterization of idea goodness and its impact on ownership bias should be investigated. Even with these limitations in mind, the results of the current investigation add to our understanding of selection biases present during the design process and provides a foundation for future work that aims to increase the effectiveness of the concept selection process in engineering design.

79 CHAPTER 6

DEVELOPING A SCALE FOR ASSESSING PREFERENCES FOR CREATIVE CONCEPT SELECTION IN ENGINEERING DESIGN

The previous chapter of this dissertation highlighted the occurrence of cognitive biases in informal concept selection processes. Specifically, this research brought to light ownership bias and gender differences that impact the objective decision-making process. The current chapter extends this work by identifying further underlying factors that may impact the concept selection process in engineering education. This is important because the informal concept selection process has the potential to be greatly affected by inadvertent decision-making biases and preferences (Baddeley, A., 2003; Miller, G.A., 1956) that can play a crucial role in shaping the creativity of the design process Therefore, research is needed that identifies the theoretical basis of preferences for creativity in design can shed light on the decision-making process in engineering design and allow for training the next generation of engineers to engage in creative concept selection. The increased focus and understanding of creativity during the concept selection process can help increase creativity throughout all stages of the design process and improve design outcomes. Thus, the current chapter was developed to address these research gaps and to identify the cognitive biases and individual attributes that influence decision-making and creativity during the informal selection process (Objective 2 of this dissertation). In addition, this chapter serves as a first step in developing a framework for measuring creative concept selection in engineering education (Objective 3 of this dissertation). The current chapter introduces a research study that was conducted to identify the underlying factors that impact creative concept selection in engineering education and develop a psychometric scale for measuring preferences for creativity. This was accomplished through a study conducted with 280 engineering students. Exploratory Factor Analysis and Confirmatory Factor Analysis were used to develop the 23-item Preferences for Creativity Scale (PCS). The results of this chapter provide a framework for conceptualizing

preferences for creativity during concept selection and allow for further research that uses standardized tools for assessing creativity during this stage of the design process.

6.1 | THE PREFERENCE FOR CREATIVITY SCALE (PCS) DEVELOPMENT

The Preferences for Creativity Scale (PCS) was developed to measure preferences for creativity during the concept selection process in engineering design. In total, 23 related concepts were used to generate an initial inventory of 116 item statements. These statements required Likert- scale responses from 1 (very inaccurate) to 5 (very accurate), see Figure 21. The complete listing of these items can be found in Appendix D: Psychometric Scale Items. The items from the PCS were derived from related research in the fields of psychology, behavioral economics, and cognitive science and are broadly categorized as research on (1) Personal Biases and Cognitive Style, (2) Creative Confidence and Motivation, and (3) Social Effects and the Environment, see Table 10. Research in these 3 themes serve as a first step in identifying the factors that can influence individual preferences for creativity, and provide a foundation for creating a framework that encompasses the factors that can impact preferences for creativity during concept selection in engineering design.

Figure 21: Example survey question in the pcs for assessing creative confidence.

Table 10: Three themes of concepts that have been identified by prior work to affect preferences for creativity.

Personal Biases and Cognitive Style Bias Against Creativity (Rietzschel, E., et al., 2010) Negativity bias (Amabile, T., 1983; Amabile, T. & Glazebrook, A.H., 1982) Optimism (Lovallo, D.P. & Sibony, O., 2010) Hindsight bias (Christensen-Szalanski, J.J. & Willham, C.F., 1991) Receptivity of new ideas (Norman, K., 1960) Ambiguity aversion (Heath, C. & Tversky, A., 1991; Sorrentino, R. & Roney, C.J.R., 2000) Cognitive way of finding answers (Guilford, J.P., 1957) Risk preference (Sitkin, S.B. & Pablo, A.L., 1992) Risk Inertia (Sitkin, S.B. & Pablo, A.L., 1992) Outcome history (Thaler, R.H. & Johnson, E.J., 1990) Burden of proof (Mounarath, R., et al., 2011) Creative Confidence and Motivation Self-efficacy for identifying and selecting creative ideas (Coopersmith, S., 1967) Creative confidence/ purpose (Phelan, S. & Young, A.M., 2003) Genuine sense of self and confidence (Rhodes, M., 1961) Intrinsic motivation to perform the task (Andrews, J. & Smith, D.C., 1996) Social Effects and the Environment Learning culture and creative climate (Ismail, M., 2005) Sensitivity to criticism (Sternberg, R.J., et al., 1997) Freedom to express opinions (Hoffman, L.R. & Maier, N., 1961) Competition (Paulus, P.B., 2000) Fear of rejection and failure (Paulus, P.B., 2000) Opinion of others/ assessment by peers (Sternberg, R.J., et al., 1997) Assessment by superiors (Davies, T., 2000) Network/ Team centrality (Perry-Smith, J.E., 2006)

6.1.1 | PERSONAL BIAS AND COGNITIVE STYLE

The first domain of concepts included in the PCS was Personal Bias and Cognitive style. Within this domain, bias against creative ideas has been shown to play an important role in preferences for creativity (Ford, C.M. & Gioia, D.A., 2000; Staw, B.M., 1995). This bias is said to occur because individuals tend to be more uncertain about whether a novel idea is practical, error- free, or useful (Amabile, T., 1996). This concept was assessed in the PCS using 4 item statements: (1) I prefer creative designs over conventional designs, (2) I believe that creative designs will lead to positive design outcomes, (3) I prefer conventional designs over creative ideas, and (4) I am skeptical that creative designs will lead to positive design outcomes.

Researchers have also shown that risk is important for making decisions under uncertainty, inherent in many real-life situations (Heath, C. & Tversky, A., 1991), and is an essential element of creativity due to its ability to encourage the individual to push boundaries and explore new territories (Kleiman, P., 2008). In fact, recent research in engineering design has identified risk and ambiguity aversion as important factors that influence preferences for creativity in the design process (Toh, C. & Miller, S., 2014; Toh, C.A. & Miller, S.R., 2016, 2016). Therefore, an individual’s level of risk preference was captured in the PCS using 2 item statements: (1) I prefer taking risks during design projects, and (2) I avoid taking risks during design projects. Related to risk-taking, another important bias is the Burden of Proof bias, characterized as a decision-making bias that arises from the individual’s need for substantial evidence or proof that a choice will result in positive outcomes, even when such information is unattainable (Mounarath, R., et al., 2011). This causes individuals to delay decision-making and refrain from acting due to the unattainable proof of positive outcomes (Mounarath, R., et al., 2011). This bias was captured using 2 item statements: (1) I am easily convinced that risky design concepts will be successful, and (2) I require proof or substantial evidence that a risky design concept will be successful before taking risks. Other concepts under the domain of Personal Biases and Cognitive style that were included in the development of the PCS include risk inertia (Sitkin, S.B. & Pablo, A.L., 1992), negativity bias (Amabile, T. & Glazebrook, A.H., 1982), hindsight bias (Christensen-Szalanski, J.J. & Willham, C.F., 1991), optimism (Lovallo, D.P. & Sibony, O., 2010), ambiguity aversion (Heath, C. & Tversky, A., 1991; Sorrentino, R. & Roney, C.J.R., 2000) and receptivity of new ideas (Norman, K., 1960) all of which have been shown to affect an individual’s perception, and thus, preferences for creativity.

6.1.2 | CREATIVE CONFIDENCE AND MOTIVATION

The second domain of concepts included in the PCS include concepts related to Creative Confidence and Motivation. Researchers have shown that attitudes toward creativity can be strongly influenced by a conscious preference and focus on creativity (Phelan, S. & Young, A.M., 2003; Rietzschel, E., et al., 2010). Thus, Creative Confidence and Purpose was captured in the PCS using 8 item statements: (1) I believe that I am a creative individual, (2) I tend to pay more attention to creative ideas, (3) I tend to favor creative ideas, (4) I intend to increase the creativity of the design process, (5) I do not believe that I am a creative individual, (6) I tend to disregard creative

ideas during the design process, (7) I tend to not think highly of creative ideas, and (8) I am not focused on increasing creativity during the design process. Researchers have also shown that self-efficacy for identifying and selecting creative ideas, or the extent to which an individual believes that they are capable of and intend to bring novel, original, and creative ideas into being is a crucial factor for creativity in the workplace (Phelan, S. & Young, A.M., 2003). This concept was captured using 2 item statements: (1) I am confident in my ability to identify when ideas are creative, and (2) I am not confident in my ability to identify when ideas are creative. Closely linked to this factor is an individual’s sense of self confidence since researchers have shown that belief in oneself is essential for creative activities (Rhodes, M., 1961). In addition, the extent to which an individual prefers creative ideas is impacted by their intrinsic motivation to perform the task and to produce creative ideas (Andrews, J. & Smith, D.C., 1996). Thus, these concepts were captured in the PCS using 6 items: (1) I am motivated to solve design problems, (2) It is easy for me to stay focused on the task at hand during a design project, (3) I feel personally invested in the success or failure of the outcomes in a design project, (4) I am not motivated to solve design problems, (5) It is hard for me to stay focused on the task at hand during design projects, and (6) I am not personally invested in the success or failure of the outcomes in a design project.

6.1.3 | SOCIAL EFFECTS AND THE ENVIRONMENT

The third domain of concepts included in the PCS was Social Effects and the Environment. A wealth of research has been conducted that investigates the influence of other external or interpersonal factors on creativity in small team settings. Specifically, prior research has shown that designers’ freedom to express their opinions and emphasis placed on others’ opinions has a tremendous potential to impact preferences for creativity in design (Davies, T., 2000; Hoffman, L.R. & Maier, N., 1961; Sternberg, R.J., et al., 1997). Researchers have argued that an individual’s perceived freedom to present ideas within their teams is crucial for creative problem solving in teams (Hoffman, L.R., et al., 1962). That is, if dominance by one or multiple members of a team interferes with the free expression of ideas and opinions, the emergence of creative solutions to the problem will be dramatically reduced (Hoffman, L.R., et al., 1962). This was captured in the PCS using 4 item statements: (1) I feel comfortable presenting my ideas to my team members, (2) I do not feel judged by my team members for my ideas and opinions, (3) I do not feel comfortable

presenting my ideas to my team members, and (4) I feel judged by my team members for my ideas and opinions. Other factors such as team centrality (Perry-Smith, J.E., 2006), or the amount of influence held by an individual in a team has been shown to influence an individuals’ willingness to be creative in a team context. This is because individuals are more willing to invest in creativity within a team if they believe that they will significantly influence the direction of the project. Therefore, Team Centrality was assessed in the PCS using 4 items: (1) I play a central role in teams that I am a part of, (2) I believe that I influence the direction and progress of projects that I am a part of, (3) I do not typically play a central role in teams that I am a part of, and (4) I believe that I have little impact on the direction and progress of projects that I am a part of. In addition, an individual’s willingness to take social risks and face the possibility of criticism, rejection, and failure, is a crucial component of preferences for creativity (Paulus, P.B., 2000; Sternberg, R.J., et al., 1997), since a majority of design work is conducted in small teams. External influences such as learning culture and creative climate (Ismail, M., 2005) have also been shown to be essential for fostering creativity in teams. This concept was captured using 2 items: (1) I feel that my current working environment encourages and fosters creativity, and (2) I feel that my current working environment does not encourage or foster creativity.

85 6.2 | METHODOLOGY

As a first step at addressing Objective 3 of this dissertation, a study was developed to address the following sub-objectives: (3a) identify and test the internal consistency of the underlying factors of preferences for creativity during engineering education concept selection; and (3b) develop and validate a scale for measuring preferences for creativity during concept selection in engineering education.

6.2.1 | PARTICIPANTS

A total of 280 engineering students (203 males, 77 females) were involved in this experiment. Data collection occurred in 2 phases, over the course of 2 academic years. For the first round of data collected, 106 first year engineering students (84 males, 22 females) were recruited from 4 sections of an introduction to engineering design course at Penn State (EDSGN100) in the same semester. One year after the first round of data collection, an additional 174 students (119 males, 55 females) were recruited from 2 sections of an introduction to engineering design course (EDSGN100, N = 62), 1 section of an entrepreneurial leadership course (ENGR 310, N = 22), 1 section of a technology-based entrepreneurship course (ENGR 407, N = 28), 2 sections of a mechanical design methodology course (ME 340, N = 50), and 1 section of an industrial engineering capstone course (IE 480W, N = 12). This second round of data collection utilized a larger sample size with more diverse backgrounds in order to verify the validity of the PCS in broader a population.

6.2.2 | PROCEDURE

During the study, participants provided informed consent, and were asked to complete a 116-item Preference for Creativity Scale (PCS) that focused on individual traits, attitudes, and behaviors that can influence preferences for creativity during concept selection, see PCS Survey

Development for details. Participants were given a week to complete the survey using an online survey form.

6.3 | EXPLORATORY FACTOR ANALYSIS

To identify the underlying factors that influence preference for creative ideas during concept selection and to develop a scale for measuring preference for creative concept selection, an Exploratory Factor Analysis (EFA) (Hatcher, L. & Stepanski, E., 1994) was performed on the responses to the 116 questions of the Preference for Creativity Scale (PCS) from the first round of data collection with 106 first-year engineering students. Analysis was conducted using SPSS v. 22.0 (IBM Corp., I., Released 2013). Negatively-worded items were reverse coded before analysis and only items that demonstrated high primary factor loadings (0.50 or above) and low cross-factor loadings were selected for analysis (McMurray, A., et al., 2004). The Principle Components Analysis with Varimax rotation resulted in 23 items loading on 4 factors, see Table 10. All of the variables included in this model had communalities greater than 0.20 and the items accounted for 55% of the variance found in the model. While communalities of 0.2 are considered relatively low and indicate that there is considerable variance not explained by the factors (Valicer, W.F. & Fava, J.L., 1998), only 1 item (“I do not care about getting good grades”) obtained this value of communalities. Therefore, even though low communalities indicate that factors could be added to increase the amount of variance explained by the model, fewer factors were chosen for this analysis in order to maintain model parsimony and simplicity. The first factor in the resulting model was titled Team Centrality and Influence due to its focus on assessing team dynamics and the influence of the individual in the team on preferences for creativity. This factor accounted for 18% of the variance in the model and had a high level of internal consistency, as determined by a Cronbach’s alpha of 0.85. Constructs in this factor included items such as “I play a central role in teams that I am a part of” and “I believe that I influence the direction and progress of projects that I am a part of”. The basis of this factor is supported by prior research that has shown that team centrality (Perry-Smith, J.E., 2006), freedom to express opinions (Hoffman, L.R., et al., 1962), sensitivity to criticism (Sternberg, R.J., et al., 1997), and fear of

failure (Paulus, P.B., 2000) all play central roles in affecting an individual’s willingness to present and select creative ideas. The second factor was titled Risk Tolerance due to its focus on the influence of tolerance of risk or failure on preferences for creativity during the design process. This factor accounted for 16% of the variance in the model and had a high internal consistency as determined by a Cronbach’s alpha of 0.84. Examples of items in this factor include “I believe that risky design concepts will lead to positive design outcomes” and “I do not like dealing with ambiguous or unknown elements in the design process”. The influence of risk and ambiguity aversion is supported by prior work in related fields that have shown that tolerance for risk is important for making decisions under uncertainty, inherent in many real-life situations (Heath, C. & Tversky, A., 1991). The willingness to take risks has been shown by prior research to be an essential element of creativity due to its ability to encourage the individual to push boundaries and explore new territories (Kleiman, P., 2008). In addition, risk inertia, or an individual’s tendency to take risks based on prior risk behavior, has been shown to play an important role in risk-taking behavior (Sitkin, S.B. & Pablo, A.L., 1992). Similarly, outcome history is important in influencing current risk behavior, since individuals who have experienced positive outcomes when taking risks are more likely to take risks again, and vice versa (Thaler, R.H. & Johnson, E.J., 1990). The third factor was called Creative Confidence and Preference due to its focus on assessing the extent to which the individual prefers and gives priority to creativity during the design process and possesses confidence in bringing these ideas to life. This factor accounted for 15% of the variance in the model and had a high internal consistency as determined by a Cronbach’s alpha of 0.81. Constructs in this factor included items such as “I prefer creative designs over conventional designs” and “I intend to increase creativity during the design process.” This is supported by prior research that has shown that conscious preference and focus on creativity is important for encouraging creativity in design (Phelan, S. & Young, A.M., 2003; Rietzschel, E., et al., 2010). In addition, researchers have shown that the extent to which an individual believes that they are capable of and intends to bring a novel, original, and creative idea into being is a crucial factor for creativity in the workplace (Phelan, S. & Young, A.M., 2003).

88 Table 11: Factor structure of preferences for creative ideas during concept selection (higher means indicate a stronger preference for creative ideas). Items labeled with (N) indicate negatively-worded items that were reverse coded for analysis.

89 The final factor was titled Motivation due to its focus on the individual’s level of investment and motivation in completing the task at hand. This factor was only moderate compared to other factors accounting for only 6% of the variance in the model with a Cronbach’s alpha of 0.54. Examples of items in this factor include “I do not care about getting good grades” and “I am not easily discouraged when I am being criticized.” Previous research has argued that since creativity requires significant effort and concentration, personal investment in projects and encouraging environments are required for creativity (Andrews, J. & Smith, D.C., 1996). The low reliability shows that items in this factor may not be highly related to one another, and may be a secondary factor of preferences for creativity in engineering design students.

6.4 | CONFIRMATORY FACTOR ANALYSIS

Once an exploratory factor structure was developed, Confirmatory Factor Analysis (CFA) was conducted to validate the results and finalize the Preferences for Creativity Scale (PCS). Responses to the PCS from the second round of data collection with the 174 engineering students from various levels and classes was used for this analysis using AMOS v. 22.0 (Arbuckle, J.L., 2006). The four-factor structure obtained in EFA was directly specified in CFA, and each item was restricted to load onto one factor only, see factor structure in Figure 22. Once again, negatively- worded items were reverse coded before analysis, and the CFA was conducted on the results of the PCS.

Figure 22: Factor structure of the 4-factor model with added error correlations.

The results revealed that the initial factor model was not satisfied by strict criteria such as the chi-square test of exact fit (χ2(226)=551.53, p < 0.001). In addition, approximate measures of fit indicate unsatisfactory model fit (standardized root mean square residual = 0.095) (Hair, J.F., et al., 2005). However, local fit indices indicated that modifications to the specified model would improve fit. Therefore, several modifications were implemented. Specifically, correlations were added between the error terms for 13 items in the model, see Figure 22 for modified model. These correlated error terms indicate that the variance not explained by the theoretical constructs in this model can co-vary across items and that there may be relationships between these items in the scale (Brown, T.A., 2015). Correlated error terms were only added to items within the same factor since this can prevent over-fitting of data and maintain theoretical validity of the scale (Brown, T.A., 2015). These modifications produced a four-factor model with improved fit. The approximate model fit indices indicated good model fit (standardized root mean square residual = 0.069) (Hair, J.F., et al., 2005). These results show that even though stringent test criteria such as the chi-square test of exact fit were still not met (χ2(217) = 382.6, p < 0.001), the values of approximate model fit indices that fall well below recommended thresholds commonly used in scale development (Shah,

J.J., et al., 2009) indicate that the modified model was still a reasonable measure for assessing the extent to which engineering students prefer creative ideas. The standardized loadings from the modified model are shown in Table 11. Once the model fit and factor loadings were computed for the CFA model, correlations among all items were investigated. We found no strong correlations between individual measures, indicating that all measures are capturing distinct elements of preferences for creativity, and no measure should be dropped from the scale. The average score of all 267 participants on each factor of the PCS was also computed: Team Centrality and Influence (M = 3.95, SD = 0.69), Risk Tolerance (M = 3.23, SD = 0.62), Creative Confidence and Preference (M = 3.67, SD = 0.64), Motivation (M = 4.02, SD = 0.65).

Table 12: Factor loadings of the modified four-factor model specified in the CFA. (N) indicate negatively-worded items that were reverse coded for analysis.

93 6.1 | CHAPTER SUMMARY AND DISCUSSION

The third objective of this dissertation was to develop a framework for measuring the underlying factors that impact preferences for creative ideas during concept selection. As a first step towards this objective, the current chapter identified the main dimensions of preferences for creativity by studying a total of 280 engineering students from various levels and courses at Penn State. The results of this chapter highlight the following:

• Preferences for creativity among engineering students can be broken down into 4 major dimensions: (1) Team Centrality, (2) Risk Tolerance, (3) Creative Confidence/ Preference, and (4) Motivation.

• The 23-item Preferences for Creativity Scale (PCS) is a reasonable measure for assessing the extent to which engineering students prefer creative ideas during the design process.

Specifically, one of the main findings of this chapter was that engineering students’ preferences for creativity during concept selection can be conceptualized as 4 primary dimensions: (1) Team Centrality, (2) Risk Tolerance, (3) Creative Confidence/ Preference, and (4) Motivation. The resulting 4-factor model obtained from the factor analysis differs from the broad classification used in Section 6.1 | indicating that preferences for creativity in a design context may have a different underlying structure compared to other related factors. This work provides a foundation for understanding the factors that can impact preferences for creativity during engineering design concept selection, and importantly, proposes a dimensional structure for studying and investigating preferences for creativity during the design process. Another important finding of this chapter is that team-level factors were most dominant in the model specifying preferences for creativity during engineering design concept selection. While decision-making biases (Heath, C. & Tversky, A., 1991; Sorrentino, R. & Roney, C.J.R., 2000) and creative confidence (Phelan, S. & Young, A.M., 2003; Rietzschel, E., et al., 2010) can play an important role in influencing preferences for creativity, factors that described an individual’s

relationship with their team members were the most crucial factors in determining preference for creativity during concept selection. This is important since design is considered an inherently collaborative process (Bucciarelli, L.L., 1988) that involves intricate communication patterns and roles that inadvertently impact the design process (Heath, T., 1993). Therefore, it follows that given a base level of preference for creativity among team members, social factors have the potential to greatly influence an individuals’ willingness to take risks and pursue creative alternatives in the design process. After team-level factors, Risk Tolerance, or an individuals’ willingness to take risks, was the second most dominant factor that impacted preference for creativity in this chapter. This result is supported by prior research in the field of engineering design that has identified risk and ambiguity aversion as important factors that influence preferences for creativity in the design process (Toh, C. & Miller, S., 2014; Toh, C.A. & Miller, S.R., 2016, 2016). The inherent uncertainty present in many real life situations also necessitate a certain degree of risk tolerance in order to pursue creative solutions that may not be successful (Heath, C. & Tversky, A., 1991). This research highlights risk as an important component of creativity due to its ability to encourage the individual to push boundaries and explore new territories (Kleiman, P., 2008). While the development of the PCS as a tool for assessing preferences for creativity in designers is still in its preliminary stages, initiatives within engineering education for increasing creativity during the design process in order to improve design training can be implemented. Increased focus on creativity throughout all stages of the design process can not only improve design outcomes, but can also improve awareness of the factors that can impact individual perceptions and preferences for creativity during the design process. For example, even though creativity is heavily emphasized in the engineering classroom (Litzinger, T.A., et al., 2011; Richards, L.G., 1998; Stouffer, W.B., et al., 2004; Sullivan, J.F., et al., 2001), students are often risk averse during design projects (Linnerud, B. & Mocko, G., 2013). By educating students on the various decision-making biases and common thinking patterns that can impact openness to creativity, students can begin developing skills for recognizing and selecting creative ideas during the design process. In addition, research focused on preferences for creativity can also encourage design educators to emphasize creativity equally during idea generation and concept selection, and to include creativity as an important component of engineering education (Daly, S.R., et al., 2014). The results of this chapter also highlight the need to focus on creating team environments that foster freedom of expression and openness to creativity during the design process. Since team- level factors play a dominant role in preferences for creativity during the design process, engineering design students should be encouraged to communicate and share ideas freely in order

to promote creativity during concept selection (Davies, T., 2000; Hoffman, L.R. & Maier, N., 1961; Sternberg, R.J., et al., 1997). In addition, discussion regarding the important role that each team member plays in a design team and methods of meaningfully contributing to the success of the design project can help to increase team functionality and creative potential (Perry-Smith, J.E., 2006). Importantly, since students’ prior experiences with risk taking was identified as an important factor of preferences for creativity, educational settings and assessment methods that encourage risk-taking and creativity can help to increase risk tolerance in engineering students in the future. While intrinsic validation of the scale has been conducted in this chapter, full validation, including an investigation of criterion and construct validity need to be conducted in future studies using large amounts of data and by comparing the PCS with other related tests. In addition, detailed reliability analysis of the items in the scale should be conducted in order to investigate the effectiveness of each item. Survey responses from industry professionals will help expand the generalizability of the scale and provide an understanding of how engineering students and professionals differ in terms of preferences for creativity. Future work that explores preferences for creativity longitudinally and in different contexts and domains will also help shed light on the factors that impact designers’ attitudes toward creativity in order to develop methods and technique that foster creativity throughout the design process.

96 CHAPTER 7

INVESTIGATING THE IMPACT OF PREFERENCES FOR CREATIVE CONCEPT SELECTION ON DESIGNER BEHAVIOR

The results of the previous chapter identified the primary factors that constitute preferences for creativity in engineering design and provided the validated Preferences for Creativity Scale (PCS). While there exists research that has shown that factors such as risk attitudes (Kleiman, P., 2008), social effects (Perry-Smith, J.E., 2006), creative confidence (Phelan, S. & Young, A.M., 2003), and motivation (Andrews, J. & Smith, D.C., 1996) can impact creativity and decision- making, research that investigates the role of these factors on creative concept selection in engineering design is sparse. Furthermore, the impact that preferences for creativity can have in a team setting is important to investigate since design is increasingly being recognized and taught as a team process in engineering (Dym, C.W., JW; Winner, L, 2003). Therefore, further research is needed to establish an empirical link between the 4 theoretically derived factors in the previous chapter with actual designer behavior during a creative concept selection activity. Thus, as a second step towards addressing Objective 3 of this dissertation, the current chapter was developed to empirically investigate the impact of the PCS on the behavior and creativity of designers during a design activity, above and beyond the effects of personality traits since traits have been shown to heavily influence risk-taking, team performance, and creativity (Nicholson, N., et al., 2005; Somech, A. & Drach-Zahavy, A., 2011; Wilde, D.J., 1997; Zuckerman, M. & Kuhlman, D.M., 2000). This was accomplished through an empirical experiment with 178 engineering students at Penn State. The results of this chapter link the PCS factor scores with creative concept selection in engineering education and provide an understanding of the factors that can affect creativity and decision-making during engineering design concept selection.

7.1 | METHODOLOGY

In order to address Objective 3 of this dissertation, an empirical study was developed to address the following research sub-objectives: (3c) understand if the PCS factors obtained in Chapter 6 are related to the creativity of generated ideas above and beyond the effects of personality traits, and (3d) understand if the PCS factors obtained in Chapter 6 are related to the creativity of ideas selected by participants above and beyond the effects of personality traits.

7.2 | PARTICIPANTS

To address these research sub-objectives, a total of 178 engineering design students were drawn from the same participant pool as Chapter 6 of this dissertation. There were a total of 100 first year engineering students (78 males, 22 females) recruited from 4 sections of an introduction to engineering design, and 78 first year and third year students (57 males, 21 females) recruited from 2 sections of an introduction to engineering design course and 2 sections of a mechanical design methodology course at Penn State. Participants in each course were assigned to 3 or 4- member design teams by the course instructor at the start of the semester.

7.3 | PROCEDURE

The experiment was conducted in 2 phases involving data collected from the PCS and a design session comprising of idea generation and selection activities, see Figure 23.

Figure 23: Experiment timeline depicting 2 distinct phases, where participants complete online surveys 1 week prior to the design activities.

First, participants were asked to complete the Preference for Creativity Survey (PCS) that focused on individual traits, attitudes, and behaviors that can influence preferences for creativity during concept selection, see Chapter 6 for details on the development of the PCS. In addition, personality measures for each participant were captured using the MINI-IPIP scale (Donnellan, M.B., et al., 2006), a short form of the Five Factor Model Measure of personality (Goldberg, L.R., 1999). This short form was chosen for this study in order to minimize fatigue and maximize response rates in participants while completing online surveys with many items (Galesic, M. & Bosnjak, M., 2009). One week after the surveys were completed, an in-class design session was conducted. At the start of this session, the researchers introduced the outline of the day’s activities using a verbal script and any questions were answered. Next, participants were provided with one of the following four design goals via written instructions on individual sheets of paper:

Milk frother (13 teams): “Your task is to develop concepts for a new, innovative, product that can froth milk in a short amount of time. This product should be able to be used by the consumer with minimal instruction. Focus on developing ideas relating to both the form and function of the product.”

Urinary tract infection (UTI) test strip (9 teams): "Your task is to develop concepts for a new mechanism to expose test strips to urine samples. This product should be simple, inexpensive, low-waste, and durable and constructed of locally-available materials."

Greenhouse grid (14 teams): "Your task is to develop concepts for a new tool to determine levelness of ground in a 7x7 meter grid for a 6x6 meter greenhouse, and to mark 49 frame post locations which are square. Any one post can be no more than 1 centimeter off and the grid should be completely marked in no more than 10 minutes. The device to measure levelness

should be lightweight and ruggedized for the harsh environment with a budget of $10. The materials are limited to nylon string, wood, and metal bars."

Reducing Pedestrian Accidents (N =22): “Your task is to develop concepts for a new, innovative product or system that will reduce pedestrian accident rates due to distraction from mobile devices. Your ideas should reduce the number of accidents on college campuses due to the increased usage of mobile devices (listening to music, texting, and talking) all of which are distracting.”

These four design tasks were selected for the analysis in this chapter due to their varying structure and domain of knowledge. Specifically, the milk frother design task was considered relatively structured, and addressed a problem in the electro-mechanical appliances domain. On the other hand, the urinary tract infection and greenhouse grid design problem were considered relatively open-ended but with considerable constraints, and addressed a problem in the mechanical, humanitarian engineering and sustainable design domains. Lastly, the pedestrian accidents design problem was very open-ended, and included solutions of physical devices as well as environmental changes. Thus, these 4 different design problems were used in this chapter to examine the effects of preferences for creativity in various solution spaces in order for the results to generalize to more than a single, isolated design task. Once the design tasks was distributed and read by the participants, they were given the opportunity to ask questions. The design task and idea generation procedures used in this chapter were the same methods described in Section 3.1.2 | Similar to in the prior chapters, following the idea-generation session, participants returned for a second design. Participants were asked to review and assess all concepts that their design team had generated in the previous session. The individual concept evaluation procedure employed in Section 3.1.3 | of this dissertation was utilized for this activity.

100

7.4 | METRICS

Once the experiment was complete, two independent raters were recruited to assess the creativity of the 1332 ideas generated based on Shah, J.J., et al. (2003) novelty and quality metrics. Specifically, the following metrics were computed according to methods described in Section 3.1.4 | and Section 4.1.3 | of this dissertation: Design Novelty, Design Quality, Participant Task-Related Novelty, Participant Task-Related Quality, and the Five-Factor Model Personality Traits. All metrics were calculated at the individual level for this chapter. In addition, the following metrics were developed and used in this chapter:

Novelty of Selected Ideas: This metric was developed to capture the extent to which participants’ selected novel ideas during the selection process. Thus, this metric was calculated as the average novelty of all ideas selected by each participant.

Novelty of Ideas not Selected: This metric was developed to capture the extent to which participants filtered out novel ideas during the selection process. Thus, this metric was calculated as the average novelty of all ideas not selected by each participant out of the ideas generated by the whole design team.

Quality of Selected Ideas: This metric was developed to capture the extent to which participants selected quality ideas during the selection process. Thus, this metric was calculated as the average quality of all ideas selected by each participant.

Quality of Ideas not Selected: This metric was developed to capture the extent to which participants filtered out quality ideas during the selection process. Thus, this metric was calculated as the average quality of all ideas not selected by each participant out of the ideas generated by the whole design team.

101

7.5 | DATA ANALYSIS AND RESULTS

The purpose of this chapter was to explore the impact of the PCS factors on participant behavior in a design context. Thus, this chapter aimed to determine the influence of the PCS measure on the creativity of ideas generated by participants above and beyond confounds and other previously identified factors that can impact idea generation such as personality traits. To accomplish this, participants’ responses to the PCS were saved in the form of standardized factor scores corresponding to each factor identified in Section 6.3 | using the Regression approach. Prior to conducting analyses to address the research sub-objectives of this chapter, preliminary analyses were conducted on the data. In order to investigate the potential impact of confounds such as the design task used for the activity and participant gender, an ANOVA was conducted using design task as the independent variable (milk frother, UTI Test Strip, Greenhouse Grid, Pedestrian Accidents design problems), and the novelty of ideas selected during the design task as the dependent variable. Since Levene’s Test for Equality of Variances showed that the variances between groups were unequal due to differing group sizes, the groups were compared using an unequal variance F-test. The results revealed that the design task (F(3, 53) = 388.8, p- value < 0.001) significantly impacted the novelty of the ideas selected. Additional t-tests were also conducted to investigate the relationship between participant gender and the year in which the data was collected with the novelty of the selected ideas using an unequal variance t-test, and the results revealed that gender (t (101) = -2.33, p-value = 0.02) and data collection year (t (110) = -23.2, p- value < 0.001) did significantly impact the novelty of the selected ideas. Therefore, the subsequent analyses were conducted with design task, data collection year, and gender included as covariates in the model so as to control for their effects in the analyses. The following sections detail the data analysis and results according to our research questions.

7.5.1 | ARE HIGH QUALITY OR HIGHLY NOVEL IDEAS FILTERED OUT DURING THE CONCEPT SELECTION PROCESS?

The first sub-objective of this chapter (3c) investigated whether or not participants were selecting or filtering out creative ideas during the concept selection process. Before the impact of the PCS factor scores on creative concept selection could be examined, the process of selecting

102

creative ideas was investigated in this research question. This was addressed using a linear regression analysis on the dependent variable of the novelty of selected ideas and the independent variables of the novelty of ideas not selected, participant gender, and design task. The results revealed that the novelty of selected ideas was not significantly related to the novelty of the ideas not selected during the selection activity (B = -0.08, p-value = 0.08). A similar regression analysis was conducted on the quality of the ideas selected, and the results showed that the quality of the ideas selected by the participant had a significant but weak relationship with the quality of the ideas not selected during the selection process (B = 0.16, p-value = 0.01). These results indicate that there is no empirical link between the novelty of ideas selected during a design task, and the other ideas generated by the design team. However, participants were selecting ideas with similar quality scores compared to other ideas generated by the team. Thus, while novel idea selection can occur independent of the novelty of the generated ideas, the quality of the ideas selected for further development was largely dictated by the quality of the ideas generated by the design team as a whole. However, the relatively low values of regression coefficients for the quality of ideas not selected (B = 0.16) indicate that this relationship is not a one-to-one relationship, see Figure 24. That is, the quality of ideas selected are, on average, of lower than the quality ideas not being selected during the concept selection process. Specifically, for every unit increase in quality of ideas not selected, the expected increase in quality of the ideas selected is only approximately 16% of that. These results provide empirical evidence that participants are filtering out high quality ideas during the concept selection process by selecting ideas that are less quality than the set of ideas available to choose from.

103

Figure 24: The relationship between the quality of ideas selected and the quality of ideas not selected during the concept selection activity.

7.5.2 | ARE PCS FACTOR SCORES RELATED TO THE CREATIVITY OF GENERATED IDEAS?

The second sub-objective of this chapter (3d) focused on investigating the impact of the Preferences for Creativity Scale (PCS) factor scores on the creativity of ideas generated during the design process. Since the participant personality traits have been shown to heavily influence risk- taking, team performance, and creativity (Nicholson, N., et al., 2005; Somech, A. & Drach-Zahavy, A., 2011; Wilde, D.J., 1997; Zuckerman, M. & Kuhlman, D.M., 2000), the impact of the factor scores on the creativity of the generated ideas was investigated while taking into account the effects of the Big Five Factors of Personality. To accomplish this, a hierarchical regression analysis was conducted with the dependent variable of Task-Related Novelty. The independent variables were then entered in 3 blocks: (i) covariates including participant gender, the assigned design task (Milk Frother, UTI Test Strip, Greenhouse Grid, Reducing Pedestrian Accidents), and the year in which the data was collected (first year, second year), (ii) Big Five Factor Personality trait scores (extraversion, agreeableness, conscientiousness, neuroticism, and openness), and (iii) factor scores from the PCS. A visual schematic of the hierarchical regression analyses used in this chapter is shown in Figure 25.

104

Figure 25: Hierarchical regression analysis schematic for testing the relationship between the factor scores and dependent variables above and beyond covariates and personality traits.

The results of the hierarchical regression analysis show that the covariates of participant gender, assigned design task, and the year that the data was collected significantly predict the novelty of the ideas generated by participants (R2 = 0.67, p < 0.001). However, personality traits alone unable to predict the participant task-related novelty above and beyond the effects of these confounds (R2 change = 0.01, p = 0.54). In addition, the PCS factor scores were unable to significantly predict participant task-related novelty above and beyond the effects of personality traits (R2 change = 0.01, p = 0.39), see Table 13 for details. These results indicate that the Big Five Factors of Personality and PCS factor scores are not related to the novelty of ideas generated by participants in this chapter.

105

Table 13: Summary of the hierarchical regression analysis results with Task-Related Novelty, as the dependent variable. Bolded rows indicate significant results.

Independent Variables Unstandardized Sig. B Coefficient Gender -0.01 0.72 R2 = 0.67, Step 1 Design Task 0.02 0.07 p < 0.001 Data Collection Year 0.19 0.00 Gender -0.01 0.58 Design Task 0.02 0.07 Data Collection Year 0.19 0.00 R2 = 0.68, Extraversion 0.00 0.98 Step 2 p < 0.001 Agreeableness 0.01 0.45 Conscientiousness -0.01 0.34 Neuroticism 0.00 0.84 Openness 0.01 0.19 Gender -0.01 0.62 Design Task 0.02 0.09 Data Collection Year 0.20 0.00 Extraversion 0.00 0.97 Agreeableness 0.01 0.42 R2 = 0.69, Conscientiousness -0.01 0.18 Step 3 p < 0.001 Neuroticism 0.00 0.90 Openness 0.02 0.22 Factor 1: Team Centrality and Influence 0.00 0.93 Factor 2: Risk Tolerance 0.02 0.08 Factor 3: Creative Confidence and Preference 0.00 0.91 Factor 4: Motivation 0.01 0.41

A second hierarchical regression analysis was conducted with the same independent variable blocks, while using Task-Related Quality as the dependent variable instead. The results of the hierarchical regression analysis show that the covariates of participant gender, assigned design task, and the year that the data was collected significantly predict the quality of the ideas generated by participants (R2 = 0.04, p = 0.05). However, personality traits alone were unable to predict the participant task-related novelty above and beyond the effects of these covariates (R2 change = 0.02, p = 0.66). In addition, the PCS factor scores were unable to significantly predict participant task- related novelty above and beyond the effects of personality traits (R2 change = 0.02, p = 0.41), see Table 14 for details. These results indicate that the Big Five Factors of Personality and PCS factor scores are not related to the quality of ideas generated by participants in this chapter.

106

Table 14: Summary of the hierarchical regression analysis results with Task-Related Quality, as the dependent variable.

Independent Variables Unstandardized Sig. B Coefficient Gender 0.02 0.58 R2 = 0.04, Step 1 Design Task 0.06 0.01 p = 0.05 Data Collection Year -0.14 0.02 Gender 0.01 0.86 Design Task 0.07 0.01 Data Collection Year -0.15 0.02 R2 = 0.06, Extraversion -0.01 0.51 Step 2 p = 0.66 Agreeableness 0.03 0.16 Conscientiousness -0.01 0.67 Neuroticism -0.02 0.33 Openness 0.00 0.90 Gender 0.00 0.98 Design Task 0.07 0.01 Data Collection Year -0.14 0.03 Extraversion 0.00 0.92 Agreeableness 0.03 0.14 R2 = 0.08, Conscientiousness 0.00 0.82 Step 3 p = 0.41 Neuroticism -0.03 0.14 Openness 0.00 0.88 Factor 1: Team Centrality and Influence 0.02 0.51 Factor 2: Risk Tolerance 0.01 0.69 Factor 3: Creative Confidence and Preference 0.02 0.64 Factor 4: Motivation 0.04 0.13

7.5.3 | DO PCS FACTOR SCORES PREDICT THE CREATIVITY OF THE SELECTED IDEAS ABOVE AND BEYOND PERSONALITY TRAITS?

The third sub-objective of this chapter focused on investigating the impact of the Preferences for Creativity Scale (PCS) factor scores on the creativity of ideas selected by participants during the procedures. Similar to the previous research question, the impact of the factor scores on the average creativity of ideas selected by each participant was investigated while taking into account the effects of covariates and the Big Five Factors of Personality. To accomplish this, a hierarchical regression analysis was conducted with the dependent variable of the Novelty of Selected Ideas. The independent variables were then entered in 3 blocks: (i) covariates including participant gender, the assigned design task (Milk Frother, UTI Test Strip, Greenhouse Grid, Reducing Pedestrian Accidents), and the year in which the data was collected (first year, second

107

year), (ii) Big Five Factor Personality trait scores (extraversion, agreeableness, conscientiousness, neuroticism, and openness), and (iii) factor scores from the PCS. The results show that the covariates of participant gender, assigned design task, and the year that the data was collected significantly predict the novelty of the ideas selected during the design task (R2 = 0.71, p < 0.001). In addition, the Big Five Factors of Personality scores do not significantly predict the novelty of the selected ideas above and beyond the effects of these confounds (R2 change = 0.004, p = 0.80). When added to the covariates and the Big Five Factors of Personality scores, the PCS scores led to a modest but statistically significant increase in the amount of variance explained in the novelty of the ideas selected (R2 change = 0.02, p = 0.02). Specifically, scores for Factor 1: Team Centrality and Influence (B = 0.02, p = 0.05) and Factor 2: Risk Tolerance (B = 0.01, p = 0.04) significantly predicted the novelty of selected ideas, above and beyond the effects of covariates and the Big Five Factors of Personality, see Table 15. These results indicate that high levels of Factor 1: Team Centrality and Influence and Factor 2: Risk Tolerance led to higher levels of novelty in the ideas selected during the selection activity.

Table 15: Summary of the hierarchical regression analysis results with the Novelty of Selected Ideas, as the dependent variable. Bolded rows indicate significant results.

Independent Variables Unstandardized Sig. B Coefficient Gender 0.01 0.29 R2 = 0.72, Step 1 Design Task 0.02 0.05 p < 0.001 Data Collection Year 0.19 0.00 Gender 0.02 0.28 Design Task 0.02 0.05 Data Collection Year 0.19 0.00 R2 = 0.72, Extraversion 0.00 0.93 Step 2 p < 0.001 Agreeableness 0.00 0.93 Conscientiousness 0.00 0.55 Neuroticism -0.01 0.19 Openness -0.01 0.47 Gender 0.01 0.29 Design Task 0.02 0.13 Data Collection Year 0.20 0.00 Extraversion 0.01 0.41 Agreeableness 0.00 0.99 R2 = 0.74, Conscientiousness -0.01 0.31 Step 3 p < 0.001 Neuroticism -0.02 0.04 Openness -0.01 0.57 Factor 1: Team Centrality and Influence 0.02 0.05 Factor 2: Risk Tolerance 0.03 0.01 Factor 3: Creative Confidence and Preference 0.02 0.14 Factor 4: Motivation 0.01 0.47

108

A second hierarchical regression analysis was conducted with the same independent variable blocks, while using the Quality of Ideas Selected as the dependent variable instead. The results of this second hierarchical regression analysis show that the covariates of participant gender, assigned design task, and the year that the data was collected modestly but statistically significantly predict the quality of the ideas selected during the design task (R2 = 0.09, p = 0.001). In addition, the Big Five Factors of Personality scores alone do not significantly predict the quality of the selected ideas above and beyond the effects of these covariates (R2 change = 0.05, p = 0.13). The PCS factor scores were unable to significantly predict the quality of the ideas selected above and beyond the effects of the covariates and the Big Five Factors of Personality scores (R2 change = 0.03, p = 0.19), see Table 16 for details. However, when taken alone, Factor 2: Risk Tolerance, was able to predict the quality of the ideas selected during concept selection (B = 0.03, p = 0.03). These results indicate that the Big Five Factors of Personality and PCS factor scores are not related to the quality of ideas selected by participants in this chapter.

Table 16: Summary of the hierarchical regression analysis results with the Quality of Selected Ideas, as the dependent variable. Bolded rows indicate significant results.

Independent Variables Unstandardized Sig. B Coefficient Gender 0.00 0.94 R2 = 0.09, Step 1 Design Task 0.07 0.00 p = 0.001 Data Collection Year -0.14 0.00 Gender 0.01 0.86 Design Task 0.07 0.00 Data Collection Year -0.15 0.00 R2 = 0.14, Extraversion 0.00 0.84 Step 2 p = 0.001 Agreeableness -0.01 0.57 Conscientiousness 0.01 0.36 Neuroticism 0.00 0.99 Openness -0.04 0.02 Gender 0.00 0.94 Design Task 0.07 0.00 Data Collection Year -0.15 0.00 Extraversion 0.00 0.96 Agreeableness -0.01 0.58 R2 = 0.17, Conscientiousness 0.01 0.53 Step 3 p = 0.001 Neuroticism -0.01 0.54 Openness -0.05 0.02 Factor 1: Team Centrality and Influence -0.01 0.69 Factor 2: Risk Tolerance 0.04 0.03 Factor 3: Creative Confidence and Preference 0.04 0.13 Factor 4: Motivation 0.01 0.79

109

7.6 | CHAPTER SUMMARY AND DISCUSSION

The main purpose of this chapter was to investigate the impact of the Preferences for Creativity Scale (PCS) on designer behavior during a concept selection activity. This chapter highlights the following 2 major findings:

• The novelty of ideas selected during concept selection is independent of the novelty of ideas generated during idea generation. • Idea quality during concept selection is linked to the quality of ideas generated by the design team. • The Preferences for Creativity Scale can predict the novelty of the ideas selected during concept selection above and beyond the Big Five Factors of Personality.

One of the main findings of this chapter is that the creativity of the ideas generated during the earlier stages of conceptual design can play an important role in the creativity of ideas selected during concept selection. However, this relationship is complex in nature, and is dependent on the various aspects of creativity (novelty and quality). The results of this chapter show that idea selection novelty is not limited by the novelty of the ideas generated during previous stages, and that other factors such as personal attributes, cognitive biases, and organizational culture can play a role in increasing the novelty of the idea selection process. These results are supported by prior research that has found that people often perform poorly at selecting creative ideas during the evaluation process (Rietzschel, E., et al., 2010). In addition, the quality of the ideas selected during concept selection was found to be related to the quality of the prior ideas generated by the design team. Therefore, there is a limitation on the overall quality of the ideas selected for further development by the design team, but this relationship is not a one-to-on relationship. This finding is important because it indicates that the overall quality of the ideas generated during earlier stages of conceptual design are not being maximized during the concept selection stage, and can negatively affect the quality of design outcomes. Therefore, research that explores this disconnect between idea generation and idea selection can help provide a foundation for developing design techniques that leverage design creativity throughout all stages of the design process. The results of the analyses conducted in this chapter also provide empirical evidence for the impact of the Preferences for Creativity Scale (PCS) on the novelty of ideas selected during the

110

design process. In fact, this chapter showed that the PCS can better predict the novelty of selected ideas better than the Big 5 personality traits that have been shown to heavily influence risk-taking, team performance, and creativity (Nicholson, N., et al., 2005; Somech, A. & Drach-Zahavy, A., 2011; Wilde, D.J., 1997; Zuckerman, M. & Kuhlman, D.M., 2000). Importantly, Factor 2: Risk Tolerance of the PCS, dominates the relationship between creativity of selected ideas and all other predictors, indicating that this aspect of individual preferences plays an important role in creative concept selection. This result is supported by prior research that has shown that the willingness to take risks is an essential element of creativity due to its ability to encourage the individual to push boundaries and explore new territories (Kleiman, P., 2008). Other concepts included in this factor such as risk inertia, or an individual’s tendency to take risks based on prior risk behavior (Sitkin, S.B. & Pablo, A.L., 1992), and outcome history (Thaler, R.H. & Johnson, E.J., 1990) also play an important role in the selection of creative ideas during this process. Overall, the findings of this research show that while personality traits can influence creativity in individuals, other factors, identified in the PCS, are important determinants of creative decision-making in a design context. The findings of this research highlight the use of the PCS as a valid measure for preferences for creativity in a design setting, and provide a foundation for studying creativity during concept selection. Specifically, the use of the PCS will allow researchers to investigate the role that cognitive biases and individual attributes play on design decision-making, and motivate future research that aims to increase creativity throughout all phases of the design process. However, there are several important limitations that should be noted. First, since only 2 factors of the PCS were found to predict the creativity of generated and selected ideas, further work is needed to explore the effectiveness of the PCS and refine the items to improve generalizability and reliability in engineering education. In addition, all 4 factors of the PCS were only found to predict the novelty of the selected ideas better than the Big Five Factors of personality, leaving to question the impact the PCS has on the quality of selected ideas. Lastly, the relationship between the PCS factors and design tasks in other domains, using other selection methods should be explored in order to add to knowledge about the factors that can determine creative concept selection.

111 CHAPTER 8

CONCLUSIONS AND CONTRIBUTIONS

The process of selecting concepts that satisfy the design goal has been regarded by researchers as one of the most difficult and elusive challenges of successful engineering design (Pugh, S., 1996). This is because the direction of the final design is largely determined at this stage (Hambali, A., et al., 2009; King, A.M. & Sivaloganathan, S., 1999) since concepts that were generated in previous stages are selected and synthesized into a final solution in order to address the design goal (Ulrich, K.T., et al., 2011). In fact, researchers have estimated that concept selection process utilizes approximately 60-80% of design resources (Duffy, A.A., 1993). In an effort to aid in the concept selection process, a wealth of research in the design domain has focused on developing formalized concept selection methods, but informal methods such as design review meetings (Salonen, M. & Perttula, M., 2005), voting (Dym, C.L., et al., 2002) and stakeholder preferences (Ulrich, K.T., et al., 2011) are more often used to evaluate concepts in design industry. One of the main issues with using and teaching informal concept selection processes is they lack technical rigor and clear selection criteria, opening up the concept selection process to biases associated with human decision-making (Wells, J.D., et al., 2010). This not only reduces the objectivity of the selection process, but ultimately can influence the outcome of the final design (Cooper, S.Y. & Lucas, W.A., 2006). These biases can include biases towards self-generated concepts (Nikander, J.B., et al., 2014), individual personality traits (Kichuk, S. & Wiesner, W., 1998; Mann, R., 1959; Somech, A. & Drach-Zahavy, A., 2011), and risk-aversion (Kleiman, P., 2008). These factors are important to study since individuals and companies who select high quality and highly innovative concepts during this process increase their likelihood of product success and radical innovation, while those who select poor concepts have larger expenses including redesign costs and production postponement (Huang, H.-Z., et al., 2013). While research in the fields of psychology, behavioral economics, and management science provide a foundation for studying decision-making and creativity during concept selection, further research is needed to add to the fundamental knowledge of the factors that affect decision-making in an engineering design context.

112

Therefore, the purpose of this dissertation was to address 3 research objectives. First, this dissertation sought to develop a detailed understanding of the concept selection practices and team decision-making processes during concept selection in engineering design. Next, this dissertation focused on investigating the cognitive biases and individual attributes that impact team creative concept selection processes in engineering design. Lastly, this dissertation developed a theoretical understanding of the underlying factors that can impact creative concept selection in engineering design. This chapter provides a summary of the findings of each objective of this dissertation, highlighting the contributions of this research and the limitations and future research directions spurred by this dissertation.

8.1 | UNDERSTANDING THE CONSIDERATIONS USED TO SELECT CONCEPTS DURING INFORMAL CONCEPT SELECTION

A study was conducted in Chapter 3 of this dissertation to understand existing concept selection practices and the team decision-making process in informal concept selection. This chapter focused on investigating the design considerations used by student design teams to select concepts during informal group discussions. This chapter utilized content analysis of team discussions to shed light on the informal selection process in design. This section serves as a summary of this experiment.

8.1.1 | EXPERIMENT OVERVIEW

The first study of this dissertation, conducted in Chapter 3 explored the informal team discussion process for selecting ideas with engineering design students. The analysis was based on audio recordings of team discussions during a controlled design experiment and analyzing these discussions using content analysis in order to identify the key considerations used by student design teams to select concepts. The link between the types of discussions and the creativity of the selected designs was also investigated in this chapter. This was accomplished by assessing the creativity of each idea generated in this study using the Shah, J.J., et al. (2003) creativity metrics, and through

113

the development of the Propensity for Creative Concept Selection metric for assessing each teams’ tendency for selecting creative ideas during this process.

8.1.2 | REVIEW OF MAJOR FINDINGS

In study 1, conducted in Chapter 3 of this dissertation, the team discussion process during informal concept selection in design was explored. By observing and analyzing student design team discussions during an informal concept selection activity, a detailed understanding of the types of considerations being used during informal concept selection was obtained. The findings of this chapter added knowledge regarding the important role that technical feasibility plays in informal concept selection and highlighted the neglect of creativity during this concept selection process. The results of this chapter also provide empirical evidence for the link between certain discussion topics (idea decomposition, inspiration for new ideas) and the selection of more creative concepts during team concept selection. These results provide a foundational basis for studying creative concept selection and enables future research that investigates the factors that can affect the informal decision-making process in concept selection.

8.2 | IDENTIFY THE COGNITIVE BIASES AND INDIVIDUAL ATTRIBUTES THAT INFLUENCE DECISION-MAKING AND CREATIVITY DURING CONCEPT SELECTION

The second objective of this dissertation focused on investigating the cognitive biases and personal attributes associated with informal decision-making and their impact on creative concept selection in engineering design. To accomplish this, two chapters were devoted to understanding the impact of ownership bias on the concept selection process and to investigate the influence of risk attitudes and personality traits on the creativity of concepts selected. These chapters were based on controlled design studies with engineering design students who participated in idea generation and concept selection activities. This section serves as a summary of these chapters.

114

8.2.1 | EXPERIMENT OVERVIEW

Study 2 conducted in Chapter 4 of this dissertation explored the impact of individual attributes such as risk attitudes and personality traits on the creativity of selected ideas in engineering design. This research provided an empirical investigation of the link between risk attitudes in other domains of risk, such as the financial domain, and the creativity domain of risk. Thus, the risk attitudes and personality traits of engineering design students were assessed using standard behavioral economics and psychology tools and participants attended a design session where they were asked to generate ideas to address a design goal, and select ideas to move on in the design process through group discussions. The creativity of the ideas generated were once again assessed using the Shah, J.J., et al. (2003) creativity metrics, and the Propensity for Creative Concept Selection metric was used to investigate the link between risk attitudes and personality traits and each teams’ propensity for creative concept selection. Study 3, conducted in Chapter 5 of this dissertation, was developed to understand the impact of ownership bias on the concept selection process. Engineering students in deign teams were provided with a design task and asked to generate design ideas to address the design goal individually. Participants were then asked to individually evaluate all ideas generated by their team members and select design ideas for further consideration. The results of these individual idea assessment activities were then coded and analyzed in order to uncover any inherent bias for or against a designer’s own ideas. An idea “Goodness” metric was also developed in order to investigate the impact of the quality or value of an idea on ownership bias in concept selection. This was important to investigate to understand if designers were showing preferences for their own ideas simply because they were objectively “better” than their team members’ ideas.

8.2.2 | REVIEW OF MAJOR FINDINGS

The findings of study 2 in Chapter 4 of this dissertation show that team ambiguity aversion, agreeableness, and conscientiousness levels were related to propensity for creative concept selection. Specifically, teams that were more tolerant of ambiguity, agreeable, and conscientious were more likely to select more novel concepts during team informal concept selection activities. These results show that individual attributes can influence perceptions and preferences for creative ideas. Importantly, measures of risk attitudes in other domains, such as the financial domain, are

115

linked with creative risk taking behavior in the design domain. This chapter also showed that highly creative teams do not necessarily select creative concepts during the later stages of conceptual design. Thus, creative concept selection is a separate and distinct phase of the design process, and methods and tools for encouraging creativity during concept selection is important for increasing creativity throughout the entire design process. The findings of study 3 in Chapter 5 of this dissertation provided empirical evidence for the occurrence of ownership bias in engineering concept selection and highlighted the gender differences inherent in ownership bias. Specifically, male students showed a preference for their own ideas, whereas female students showed a preference for their team members’ ideas. Through an investigation of the prior literature, we postulated that this result could be attributed to the Halo Effect, or someone’s tendency to view other people’s contributions or ideas as more favorable than they objectively are. The results of this study also show that idea Goodness does not influence this gender effect, indicating that gender difference in ownership bias are not affected by how objectively “Good” an idea is during concept selection. These findings show that cognitive biases such as ownership bias can significantly impact the objectivity of the informal concept selection process, and have complex relationships with other factors such as gender.

8.3 | DEVELOP A FRAMEWORK FOR MEASURING CREATIVE CONCEPT SELECTION IN ENGINEERING DESIGN EDUCATION

The third and final objective of this dissertation focused on developing a theoretical understanding of the underlying factors that can influence creative concept selection in engineering design. To accomplish this, a psychometric measure, the Preference for Creativity Scale (PCS) was developed using prior literature in related fields. Two chapters were devoted to developing and validating this scale, and to investigate the role of the PCS on designer behavior during creative concept selection. These chapters utilized controlled design studies with over 200 engineering design students who completed the PCS and participated in design activities involving a variety of design tasks. This section serves as a summary of these experiments.

116

8.3.1 | EXPERIMENT OVERVIEW

Study 4, found in Chapter 6 was conducted in order to develop the psychometric measure for assessing individual preferences for creative concept selection. A thorough survey of relevant literature in other fields such as cognitive science, psychology, and behavioral economics was used to develop an initial inventory of 116 questions on a 5-point Likert scale. Two rounds of data collection were then conducted with engineering design students who completed this initial inventory. The responses were then analyzed using Exploratory Factor Analysis and Confirmatory Factor Analysis in order to determine if the obtained factor structure is a valid measure for assessing preferences for creativity in engineering design. The final study of this dissertation, study 5 in Chapter 7 was conducted in order to determine the link between the PCS and designer behavior during creative concept selection. Thus, student designers were asked to complete the PCS prior to attending a controlled design session where they completed a brainstorming activity and individual concept assessments of their team members’ designs. The creativity of the generated and selected designs were then assessed using the Shah, J.J., et al. (2003) (developed in a previous section of this dissertation) and were used to investigate the link between the PCS scores and each participants’ propensity for creative concept selection.

8.3.2 | REVIEW OF MAJOR FINDINGS

Study 4 in Chapter 6 was conducted to investigate the underlying constructs of creative concept selection in engineering design. The results of the Confirmatory Factor Analysis revealed 4 major components of the PCS: (1) Team Centrality and Influence, (2) Risk Tolerance, (3) Creative Confidence and Preference, and (4) Motivation. These factors were then validated using Confirmatory Factor Analysis that showed that this factor structure was a valid measure for assessing preference for creativity in engineering design. These results indicate that team level factors and risk attitudes dominate an individual’s preferences for creativity in engineering design, and tie together knowledge from related fields with creativity in engineering design. The results of this experiment serve as a theoretical foundation for understanding the underlying factors of preferences for creativity, and provide a validated measure for assessing preferences for creative concepts in engineering design.

117

In study 5, conducted in Chapter 7 of this dissertation, the relationship between PCS scores and propensity for creative concept selection during a design activity was investigated. The results revealed that participants were filtering out ideas with high quality during the selection process, but the novelty of selected ideas were not related to the novelty of ideas generated in earlier stages of the design process. Importantly, this chapter established an empirical link between the theoretically derived factors of the PCS with designer behavior during a concept selection activity. Furthermore, even though personality has been shown to be an important factor in determining creativity and decision-making, the PCS factors were able to predict the creativity of generated ideas and selected ideas better than personality traits. These results provide a foundation for studying creativity during concept selection and show that the Preferences for Creativity Scale (PCS) developed in Chapter 6 of this dissertation is a valid measure for assessing design students’ preferences for creativity in concept selection.

8.4 | IMPLICATIONS

The implications of this dissertation are informed by the 3 objectives stated earlier in this chapter: (1) develop a detailed understanding of the team decision-making process during concept selection in engineering design, (2) investigate the cognitive biases and individual attributes that impact the team creative concept selection processes in engineering design, and (3) develop a theoretical understanding of the underlying factors that can impact creative concept selection in engineering design. The implications for each of these objectives will be discussed in the following sections.

8.4.1 | IMPLICATIONS FOR CONCEPT SELECTION METHODS IN INDUSTRY AND EDUCATION

The introduction and related work sections of this dissertation highlighted the prevalence of informal concept selection practices in design industry and show the need for a deeper understanding of the informal selection process in design education. The results of this dissertation

118

provide insights into the informal concept selection process by showing that engineering design students are highly focused on technical feasibility during the concept selection process and tend to neglect creativity during the concept selection process. This dissertation also highlights the fact that teams who continue to act on inspiration and generate ideas during the concept selection stage of the design process tend to select more creative ideas. In terms of implications for engineering education, this dissertation shows that a re-framing and re-structuring of concept selection practice and instruction in engineering education is necessary if creative ideas are to pass through the concept selection process and ultimately add value to the design process. The results provide evidence for supporting a more streamlined and coherent conceptual design process in engineering design education that truly allows for the co- evolution of problem and solution space. This coupled approach to concept generation and selection will not only increase creativity but can also improve the flexibility and effectiveness of the design process. Thus, engineering design instruction and techniques that encourage designers to be inspired through idea generation and selection should be developed and implemented in order to improve the effectiveness of the design process and help encourage creativity. Future research should be geared at investigating the impact of modifications in educational practices on both the selection of candidate ideas and the final design idea implemented in order to better understand the impact of educational structure on concept selection. It is also crucial that we investigate the integration of these methods into the engineering design curriculum in order to ensure that student designers are able to understand and implement these methods in an applied context. It is important that the utility of these methods are demonstrated and emphasized in the classroom in order for students to fully understand the importance of these methods in practice. In addition, students should be exposed to informal methods of selecting concepts in order to increase awareness of the types of methods typically found in design practice. By introducing these methods early on in the curriculum, student designers can be trained to assess and select concepts more thoughtfully even while utilizing informal and subjective methods. Design instruction can also play a role in educating student designers about the biases present during the design process, in order to draw attention to the highly subjective nature of most concept selection methods practiced in industry. These efforts can provide student designers with the skills necessary for making objective decisions in industry and combat the individual and systemic bias against creativity.

119

8.4.2 | IMPLICATIONS FOR COGNITIVE BIASES AND INDIVIDUAL ATTRIBUTES IN CONCEPT SELECTION

The introduction and related work chapters of this dissertation also identified the need for detailed studies that investigate cognitive biases and individual attributes in the informal concept selection process. While research outside of engineering design have studied factors such as ownership bias, risk attitudes, and personality traits, there is a lack of knowledge regarding the impact that these factors can play in engineering design concept selection, and their role in creative concept selection. Therefore, this dissertation sought to investigate these factors in detail to add to our understanding of the informal decision-making process in engineering design. The results of this dissertation show that evaluations during informal concept selection may not be entirely objective, and are subject to human decision-making biases. This bias is further influenced by the effects of gender, which cause male designers to be biased toward their own ideas, and female designers to be biased against their own ideas. These biases introduce subjectivity and inconsistency in the concept selection process, especially when informal selection methods are utilized. Our results also highlight the complex interaction of ownership bias, gender, and goodness in a design setting, indicating that the individual and social psychological processes that underlie these phenomena do not operate in isolation of one another, but in a multifaceted and inter- dependent manner. In terms of engineering design in practice, since selection biases still persist despite the level of goodness of an idea, it can be concluded that ownership bias cannot be eliminated from the design process. Strategies for increasing awareness of these decision-making biases and the impact on objective concept selection should be emphasized in engineering education and practice to reduce the impact of this bias. Furthermore, since decision-making biases reduce the objectivity of the informal selection process, the adoption and training of formalized methods in design industry can play a crucial role in increasing the effectiveness of the concept selection process. Formalized selection methods that are effective and easily implemented should be developed and investigated in engineering design practice in order to reduce the biases present during the concept selection process. However, even though formalized selection methods may reduce certain cognitive biases associated with unstructured decision-making, other types of biases and subjectivity may enter decision-making during formalized selection methods. For example, anecdotal evidence from design instructors point toward students’ tendencies to “tweak” numbers even while using formalized concept selection matrices in order to favor certain champion ideas. Alternatively,

120

engineering design education that emphasizes decision-making biases during informal team discussions and design work can help increase students’ awareness of these biases, and work toward reducing their negative impact in concept selection. Systematic and rigorous training on informal concept selection techniques in addition to formal selection methods can help prepare students for design practices in industry and enable objective and effective decision-making during these informal methods. The gender differences in ownership bias found in this study also highlight the complexities of team interactions in the engineering classroom and research and teaching efforts should be conducted that aim to address the overly-negative self-evaluations of female students who do not fit in the dominant culture of engineering. In this dissertation, female students evaluated their own generated ideas as overly-negative when compared to their team members’ ideas, consistent with prior findings that show that females tend to hold low expectations of themselves on masculine tasks, leading to overly-negative evaluations of their performance (Beyer, S., 1990). Educational strategies that aim to reduce the disparity in self-esteem and self-efficacy between male and female engineering students (Hutchison, M.A., et al., 2006) should be developed and implemented in order to address gender differences in ownership bias during concept selection. The results of this dissertation also provide a better understanding of how concepts are initially screened during the design process, showing that highly creative teams do not necessarily select creative concepts. Importantly, this dissertation showed that personality traits and risk attitudes are linked to novel concept selection in design providing empirical evidence that team- level personality attributes affect a design team’s perceptions and preference for the novelty dimension of creativity. The findings showed that different personality traits influence the generated and selection of creative ideas, indicating that the types of cognitive and social factors that influence idea generation and selection are fundamentally different and involve different sets of mental processes. Thus, the formation of teams that have diverse personality traits can help ensure that creativity is encouraged throughout the design process. This notion of beneficial diversity is not novel, as it has been argued by researchers to be crucial in building teams that have high creative performance (Klein, C., et al., 2006). Efforts to build the ‘perfect’ team composed of individuals with personality traits highly associated with creativity can be seen as a practice in futility since different types of personality traits may be linked with creativity at different stages of the design process. The results of this dissertation also show that social risk attitudes play an important role in the selection of novel ideas in teams. In the studies conducted in this dissertation, new and original

121

ideas were likely viewed with skepticism in the team, likely discouraging the selection of these ideas. However, teams that are more comfortable with making decisions in ambiguous situations and who are more willing to select ideas have unknown parameters are more likely to engage in the creative process, negating the general bias against creativity in team settings. It can be seen that perceptions and attitudes toward ambiguity appear to dominate in team concept selection tasks, outweighing team attitudes toward other types of risk. The development and adoption of environments and practices that encourage student designers to embrace ambiguity and take risks can allow students to openly and feely discuss ideas can help increase team creativity.

8.4.3 | IMPLICATIONS FOR CREATIVE CONCEPT SELECTION IN ENGINEERING DESIGN

The results of Chapter 6 of this dissertation provide a foundation for understanding the factors that can impact preferences for creativity during engineering design concept selection, and importantly, proposes a dimensional structure for studying and investigating preferences for creativity during the engineering design process. Team-level factors were the most dominant in the model specifying preferences for creativity during engineering design concept selection. While decision-making biases and creative confidence can play an important role in influencing preferences for creativity, we found that factors that described an individual’s relationship with their team members were the most crucial elements in determining preference for creativity during concept selection. This is important since design is considered an inherently collaborative process that involves intricate communication patterns and roles that inadvertently impact the engineering design process. Therefore, it follows that given a base level of preference for creativity among team members, social factors have the potential to greatly influence an individuals’ willingness to take risks and pursue creative alternatives in the design process. The second most dominant element in determining preferences for creativity was an individuals’ willingness to take risks during the design process as well as their previous experience with risk-taking. This is due to the inherent uncertainty present in many real life situations that necessitate a certain degree of risk tolerance in order to pursue creative solutions that may not be successful. The results of Chapter 7 further validates the PCS developed in this dissertation and establishes an empirical link between the PCS factors and designer behavior during creative concept selection activities. The results of this dissertation show that creative concepts are being filtered out of the concept selection process, but that this process is complex and potentially subject

122

to external factors. In addition, the findings show that the PCS factor scores are related to the novelty of selected ideas and are better predictors than personality traits, which have been shown by previous studies to be important elements of decision-making and creativity. The finding that the PCS factor scores relate to the novelty of selected ideas, and not the creativity of the generated ideas further solidifies the notion that PCS is capturing elements of preferences for creativity that are unique to the selection phase of the design process. In addition, since the PCS factors only predict the novelty of ideas selected, and not quality, this result highlights the distinction between these two important constructs of engineering creativity and indicates that different individual attributes and traits may influence preferences for novelty differently than preferences for quality. While the development of the PCS as a tool for assessing preferences for creativity in designers is still in its preliminary stages, initiatives within engineering education for increasing creativity during the design process in order to improve design training can be implemented. Increased focus on creativity throughout all stages of the design process can not only improve design outcomes, but can also improve awareness of the factors that can impact individual perceptions and preferences for creativity during the design process. The results of this dissertation also highlight the need to focus on creating team environments that foster freedom of expression and openness to creativity during the design process. Since team-level factors play a dominant role in preferences for creativity during the design process, engineering design students should be encouraged to communicate and share ideas freely in order to promote creativity during concept selection. In addition, discussion regarding the important role that each team member plays in a design team and methods of meaningfully contributing to the success of the design project can help to increase team functionality and creative potential.

8.5 | LIMITATIONS AND FUTURE DIRECTIONS

While this dissertation investigated the informal selection process and provides a validated tool for measuring creative concept selection preferences, there exist several important limitations of the work presented in this dissertation. Most important is that the experiments conducted in this dissertation were developed primarily to explore engineering student’s concept selection process through the lens of creativity. Therefore, the findings and implications of these experiments only

123

apply minimally to design industry and future work should focus on studying design teams in industry to compare the results found in this chapter with design practice. In addition, the naturalistic nature of these experiments allowed the findings to be generalizable to design team activities in engineering education, but more controlled experiments with manipulations and interventions are needed to draw conclusions about the causal nature of the relationships explored in this dissertation. For example, while study 1 of this dissertation showed a link between creative concept selection and the frequencies of these discussion topics, it is not clear if the increased discussion of these topics lead to creative concept selection, or simply if teams with more propensity for creative concept selection naturally engage in more discussions surrounding these topics. Further experimental investigations on this topic will reveal more information regarding the direction of this relationship. Similarly, while the sample size of female participants was characteristic of engineering design classes and the engineering design profession, controlled laboratory studies with larger sample sizes across various design domains and disciplines should be conducted to investigate ownership bias, gender, and goodness in more detail in order to address potential confounds and uncover the exact relationship between these factors and the underlying psychological constructs that cause them in a design setting. Another important limitation of this dissertation is the use of creativity, risk, and personality metrics used in the experiments. The standard of measuring creativity in this dissertation was the Shah, J.J., et al. (2003) creativity metrics, and is subject to its own flaws and limitations (Nelson BA, Y.J., 2009). It follows that the validity of the measures of idea creativity used in this dissertation are limited by the measures of these original metrics. For example, the conceptualization of creativity as both novelty and quality indicates that these two constructs can operate separately from one another and research findings that relate to novelty do not necessarily generalize to quality. Therefore, further work is needed to investigate the impact of personal attributes and preferences on the novelty of the selection process and the quality of the selection process in order to understand the complex nature of creativity during the engineering design process. In addition, the design quality metric was scored using a 4-point and 3-point scale developed by Linsey, J.S., et al. (2011), which allows for a quick and efficient method of assessing technical feasibility and idea effectiveness, but the limited granularity of this scale impacts the quality of the statistical analyses that use the quality metric. Similarly, the use of financial and psychometric measures of risk in a design and creativity context bring to question the generalizability of these measures to different contexts and motivate the need for new measures of risk and personal attributes that are more appropriate for use in a design setting. Other measures of

124

individual as well as team-level personality traits beyond the Big 5 Factors of Personality should also be investigated for their role on creative concept selection in order to shed light on how other facets of personality impact the creative concept selection process. This dissertation also identifies and studies human decision-making biases in an informal concept selection setting, but did not explore the impact of these biases during formal concept selection activities. This is important because while formalized selection methods aim to help designers systematically select concepts during design, these formal methods are not immune to the effects of human decision-making biases. In addition, since designers often engage in informal screening tasks prior to utilizing formalized concept selection methods and techniques, the role that cognitive biases play throughout both these phases of the design process need to be investigated. Therefore, future work that investigates the occurrence of cognitive biases during design tasks that utilize formalized concept selection activities will help shed light on the design decision-making process and allow research to develop and refine formal concept selection methods that mitigate these biases. While this dissertation provided a foundation for studying and measuring preferences for creativity in design through the development of the Preferences for Creativity Scale (PCS), full validation, including an investigation of criterion and construct validity need to be conducted in future studies using large amounts of data and by comparing the PCS with other related tests. Detailed reliability analysis of the items in scale should be conducted in order to investigate the effectiveness of each item in the scale and improve minimum communality and reliability scores. Survey responses from industry professionals will help expand the generalizability of the scale and provide an understanding of how engineering students and professionals differ in terms of preferences for creativity.

125

REFERENCES

Alicke, M. D. (1985). Global Self-Evaluation as Determined by the Desirability and Controllability of Trait Adjectives. Journal of Personality and Social Psychology, 49, 1621-1630. Altshuller, G. S. (1984). Creativity as an exact science: The theory of the solution of inventive problems (Vol. 320). Luxembourg: Gordon and Breach Science Publishers. Amabile, T. (1983). Brilliant but cruel: perceptions of negative evaluators. Journal of Experimental Psychology, 19, 146-156. Amabile, T. (1996). Creativitiy in Context. Boulder, Colorado: Westview Press. Amabile, T., & Glazebrook, A. H. (1982). A negativity bias in interpersonal evaluation. Journal of Experimental Social Psychology, 18, 1-22. Amabile, T. M. (1983). The social psychology of creativity: A componential conceptualization. Journal of Personality and Social Psychology, 45, 357-376. Andrews, J., & Smith, D. C. (1996). In search of the marketing imagination: Factors affecting the creativity of marketing programs for mature products. Journal of Marketing Research, 1, 174-187. Antonsson, E. K., & Otto, K. N. (1995). Imprecision in Engineering Design. Journal of Mechanical Design, 117. Arbuckle, J. L. (2006). Amos. In (22 ed.). Chicago: SPSS. Arup, S. (n.d.). Ambuja Knowledge Initiative. In (Vol. 2014, pp. Ambuja Knowledge Initiative). Atkins, R., & Flores, N. (2015). U.S. Engineering Schools to Educate 20,000 Students to Meet Grand Challenges. In. Washington, DC: National Academy of Engineering. Ayag, Z., & Ozdemir, R. G. (2009). A hybrid approach to concept selection through fuzzy analytic network process. Computers & Industrial Engineering, 56, 368-379. doi:http://dx.doi.org/10.1016/j.cie.2008.06.011. Baddeley, A. (2003). Working Memory: Looking Back and Looking Forward. Nature Reviews, 4, 829-839. Baer, M., Oldham, G. R., Jacobsohn, G. C., & Hollingshead, A. B. (2007). The personality composition of teams and creativity: the moderating role of team creative confidence. Journal of Creative Behavior, 42, 255-282. Ball, L. J., Lambell, N. J., Reed, S. E., & Reid, F. J. (2001). The exploration of solution options in design: A'naturalistic decision making'perspective. Designing in Context, Delft University Press, Delft, The Netherlands, 79-93. Barczak, G., Griffin, A., & Kahn, K. B. (2009). Perspective: trends and drivers of success in NPD practices: results of the 2003 PDMA best practices study. Journal of Product Innovation Management, 26, 3-23. Barone, M. J., Shimp, T. A., & Sprott, D. E. (1997). Mere Ownership Revisited: A Robust Effect? Journal of Consumer Psychology, 6, 257-284. Batey, M., & Furnham, A. (2006). Creativity, intelligence, and personality: a critical review of the scattered literature. Genetic, Social, and General Psychology Monographs, 132, 355-4329. Bearman, C., Ormerod, T. C., Ball, L. J., & Deptula, D. (2011). Explaining away the negative effects of evaluation onf analogical transfer: The petils of premature evaluation. The Quarterly journal of experimental psychology, 64, 942-959.

126

Beaty, R. E., & Silvia, P. J. (2012). Why do ideas get more creative across time? An executive interpretation of the serial order effect in divergent thinking tasks. Psychology of Aesthetics, Creativity, and the Arts, 6, 309-319. Belbin, R. M. (1981). Management teams: Why they succeed or fail. London: Heinemann. Bell, S. T. (2007). Deep-level composition variables as predictors of team performance: A meta- analysis. Journal of applied psychology, 92, 595-615. Beyer, S. (1990). Gender differences in the accuracy of self-evaluations of performance. Journal of Personality and Social Psychology, 59, 960-970. Beyer, S. (1990). Gender differences in the accuracy of self-evaluations of performane. Journal of Personality and Social Psychology, 59, 960-970. Birkhofer, H., Jansch, J., & Kloverdanz, H. (2005). An extensive and detailed view of the application of design methods and methodology in industry. Paper Presented at International Conference on Engineering Design, Melbourne, Australia, August 15018 (1675). Blumenthal, A. L. (1977). The Process of Cognition. Englewood Cliffs, NJ: Prentice Hall. Bond, A. H., & Ricci, R. J. (1992). Cooperation in Aircraft Design. Research in Engineering Design, 4, 115-130. Borghans, L., Heckman, J. J., Golsteyn, B. H. H., & Meijers, H. (2009). Gender differences in risk aversion and ambiguity aversion. Journal of the European Economic Association, 7, 649- 658. doi:10.1162/jeea.2009.7.2-3.649. Bornmann, L., Rüdinger, M., & Daniel, H. (2007). Gender differences in grant peer review: A meta-analysis. Journal of Informetrics, 1, 226-238. Bossuyt, D. L., Dong, A., Tumer, I. Y., & Carvalho, L. (2013). On Measuring Engineering Risk Attitudes. Journal of Mechanical Design, 135. Bower, G. H. (1981). Mood and memory. American Psychologist, 36, 129-148. Boyle, P. A., Yu, L., Buchman, A. S., & Bennett, D. A. (2012). Risk Aversion is Associated with Decision Making Among Community-based Older Persons. Frontiers in Psychology, 3. Boyle, P. A., Yu, L., Buchman, A. S., Laibson, D. I., & Bennett, D. A. (2011). Cognitive function is associated with risk aversion in community-based older persons. BMS Geriatrics, 11, 53. Bradshaw, S. D., Stasson, M. F., & Alexander, D. (1999). Shyness and group brainstorming: Effects on productivity and perceptions of performance. North American Journal of Psychology, 1, 267-276. Brockhaus, R. H. (1980). Risk-taking propensity of entrepreneurs. Academy of Management Journal, 23. Brown, T. A. (2015). Confirmatory factor analysis for applied research. New York, NY: Guilford Publications. Bucciarelli, L. L. (1988). An Ethnographic Perspective on Engineering Design. Design Studies, 9, 159-168. Buchenau, M., & Fulton Suri, J. (2000). Experience Prototyping. Paper Presented at Designing interactive systems: processes, practices, methods, and techniques, Brooklyn, NY, August 17-19 (424-433). Burnard, P. (1991). A method for analyzing interview transcripts in qualitative research. Nurse Education Today, 11, 461-466. Busby, J. S. (2001). Practices in Design Concept Selection as Distributed Cognition. Cognition, Technology and Work, 3, 140-149. Camacho, L. M., & Paulus, P. B. (1995). The roal of social anxiousness in group brainstorming. Journal of personality and social psychology, 68, 1071-1080.

127

Carli, L. L. (2010). Gender and group behavior. In J. C. Chrisler & D. R. McCreary (Eds.), Handbook of Gender Research in Psychology. New York, NY: Springer Science and Business Media. Chan, J., Fu, K., Schunn, C., Cagan, J., Wood, K. L., & Kotovsky, K. (2011). On the Benefits and Pitfalls of Analogies for Innovative Design: Ideation Performance Based on Analogical Distance, Commonness, and Modality of Examples. Journal of Mechanical Design, 133. doi:081004. Charness, G., & Grieco, D. (2013). Individual Creativity, Ex-ante Goals and Financial Incentives. In Departmental Working Papers. UC Santa Barbara: Department of Economics, UCSB. Chernyshenko, O. S., Miner, A. G., Baumann, M. R., & Sniezek, J. A. (2003). The impact of information distribution, ownership, and discussion on group member judgment: The differential cue weighting model. Organizational behavior and human decision processes, 91, 12-25. Christensen, B. T., & Schunn, C. D. (2008). The role and impact of mental simulation in design. Applied Cognitive Psychology, 22, 1-18. Christensen-Szalanski, J. J., & Willham, C. F. (1991). The hindsight bias: A meta-analysis. Organizational behavior and human decision processes, 48, 147-168. Christiaans, H. (2002). Creativity as a Design Criterion. Creativity Research Journal, 14, 41-54. Chrysikou, E. G., & Weisberg, R. W. (2005). Following in the wrong footsteps: Fixation effects of pictorial examples in a design problem-solving task. Journal of Experimental Psychology, 31, 1134-11448. Cooper, R. G., Edgett, S. J., & Kleinschmidt, E. J. (2002). Optimizing the stage–gate process: what best-practice companies do. Research Technology Management, 45, 21-27. Cooper, S. Y., & Lucas, W. A. (2006). Developing self-efficacy for innovation and entrepreneurship: An educational approach. International Journal of Entrepreneurship Education, 4, 141-162. Coopersmith, S. (1967). The Antecedents of Self-Esteem. New York. NY: Freeman. Corp., I. (Released 2013). IBM SPSS Statistics for Windows, Version 22.0. In. Armonk, NY: IBM Corp. Costa, P., & McCrea, R. (1992). Revised NEO Personality Inventory (NEO PI-R) and NEO Five- Factor Inventory (NEO-FFI). Odessa, FL: Psychological Assessment Resources. Crowley, B. P., & Delfico, J. F. (1996). Content analysis: A methodology for structuring and analyzing written material. In: United State General Accounting Office (GAO), Program Evaluation and Methodology Division. Daly, S. R., Mosyjowski, E. A., & Seifert, C. M. (2014). Teaching Creativity in Engineering Courses. Journal of Engineering Education, 103, 417-449. Davies, T. (2000). Confidence! Its role in the creative teaching and learning of design technology. Journal of Technology Education, 12, 18-31. De Martino, B., Kumaran, D., Seymour, B., & Dolan, R. J. (2006). Frames, biases, and rational decision-making in the human brain. science, 313, 684-687. Dewett, T. (2007). Linking intrinsic motivation, risk taking, and employee creativity in an R&D environment. R&D Management, 37, 197-208. Dijksterhuis, A. (2004). Think Different: The Merits of Unconscious Thought in Preference Development in Decision Making. Journal of Personality and Social Psychology, 87, 586- 598. Dong, A. (2005). The latent semantic approach to studying design team communication. Design Studies, 26, 445-461. Dong, A., Hill, A. W., & Agogino, A. M. (126). A document analysis method for characterizing design team performance. Journal of Mechanical Design, 126, 378-385.

128

Dong, A., Lovallo, D., & Mounarath, R. (2015). The effect of abductive reasoning on concept selection decisions. Design Studies, 37, 37-58. Donnellan, M. B., Oswald, F. L., Baird, B. M., & Lucas, R. E. (2006). The Mini-IPIP Scales: Tiny- Yet-Effective Measures of the Big Five Factors of Personality. Psychological Assessment, 18, 192-203. Dorst, K., & Nigel, C. (2001). Creativity in the design process: Co-evolution of problem-solution. Design Studies, 22, 425-437. Duffy, A. A. (1993). Design co-ordination for concurrent engineering. Journal of Engineering Design, 4, 251-261. Duffy, A. A. (1993). Design co-ordination for concurrent engineering. Journal of Engineering Design, 4, 251-261. Dunne, E. (2000). Bridging the Gap Between Industry and Higher Education: Training Academics to Promote Student Teamwork. Innovation in Education and Training International, 27, 361-371. Dylla, N. (1991). Thinking Methods and Procedures in Mechanical Design. Unpublished Dissertation, Technical University of Munich. Dym, C. L., Wood, W., H., & Scott, M. J. (2002). Rank ordering engineering designs: Pairwise comparison charts and borda counts. Research in Engineering Design, 13, 236-242. Dym, C. W., JW; Winner, L. (2003). Social Dimensions of Engineering Designs: Observations from Mudd Design Workshop III. Journal of Engineering Education, 92, 105-107. Eberle, B. (1996). Scamper: games for imagination development. Waco, TX: Prufrock Press. Edmonson, A., & Roloff, K. (2009). Overcoming barriers to collaboration: Psychological safety and learning in diverse teams. In Team effectiveness in complex organizations: Cross- disciplinary perspectives and approaches (Vol. 183-208). New York, NY: Routledge/ Taylor & Francis Group. El-Murad, J., & West, D. C. (2003). Risk and Creativity in Advertising. Journal of Marketing Management, 19, 657-673. Elder, W. R. (1998). Design modeling-- a design science approach (and why industry does not use it). Journal of Engineering Design, 9, 355-371. Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191. Feist, G. J. (1998). A Meta-Analysis of Personality in Scientific and Artistic Creativity. Personality and Social Psychology Review, 2, 290-309. Feist, G. J. (1998). A Meta-Analysis of Personality in Scientific and Artistic Creativity. Personality and Social Psuchology Review, 2, 290-309. Feist, G. J. (2006). The Inflence of Personaltiy on Artistic and Scientific Creativity. In R. J. Sternberg (Ed.), Handbook of Creativity, 6th Edition (pp. 273-296). New York: Cambridge University Press. Felder, R. M., & Brent, R. (2004). The ABC'S of engineering education: ABET, Bloom's Taxonomy, cooperative learning, and so on. In ASEE. Fellner, W. (1961). Distortion of Subjective Probabilities as a Reaction to Uncertainty. Quarterly Journal of Economics, 75, 670-694. Ford, C. M., & Gioia, D. A. (2000). Factors Influencing Creativity in the Domain of Managerial Decision Making. Journal of Management, 26, 705-732. Ford, J. D., & Form, L. W. (2002). Resistance and the background conversations to change. Journal of Organizational Change, 15, 105-121. Forster, J. (2009). Cognifitve Consequences of Novelty and Familiarity: How Mere Exposure Influences Level of Construal. Journal of Experimental Social Psychology, 45, 444-447.

129

Frey, D., Herder, P., Wijnia, Y., Subrahmanian, E., Katsikopoulos, K., & Clausing, D. (2009). The Pugh controlled convergence method: model-based evaluation and implications for design theory. Research in Engineering Design, 20, 41-58. Frey, D. D., Herder, P. M., Wijnia, Y., Subrahmanian, E., Katsikopoulos, K., de Neufville, R., Oye, K., & Clausing, D. P. (2010). Research in engineering design: the role of mathematical theory and empirical evidence. Research in Engineering Design, 21, 145-151. Friedman, H. H., & Amoo, T. (1999). Rating the Rating Scales. Journal of Marketing Management, 9, 114-123. Frisch, D., & Baron, J. (1988). Ambiguity and Rationality. Journal of Behavioral Decision Making, 1, 149-157. Fu, K., Cagan, J., & Kotovsky, K. (2010). Design team convergence: The influence of example solution quality. Journal of Mechanical Design, 132. Fuge, M., Stroud, J., & Agogino, A. M. (2013). Automatically inferring metrics for design creativity. Paper Presented at ASME Design Engineering Technical Conferences, Portland, OR, August 4-7. Furnham, A., & Yazdanpanahi, T. (1995). Personality differences and group versus individual brainstorming. Personality and Individual Differences, 19, 73-80. Galesic, M., & Bosnjak, M. (2009). Effects of questionnaire length on participation and indicators of response quality in a web survey. Public opinion quarterly, 73, 349-360. Genco, N., Holtta-Otto, K., & Seepersad, C. C. (2012). An Experimental Investigation of the Innovation Capabilities of Undergraduate Engineering Students. Journal of Engineering Education, 101, 60-81. Gerber, E. (2007). Improvisation Principles and Techniques for Design. Paper Presented at Computer/ Human Interaction Conference, San Jose, CA, 28 April- 3 May (1069-1072). Gero, J. S., & Mc Neill, T. (1998). An approach to the analysis of design protocols. Design Studies, 19, 21-61. Gibbs, G. (1995). Assessing Student Centered Courses. In. United Kingdom: Center for Staff Development. Gill, H. (1990). Adoption of design science by industry-- why so slow? Journal of Engineering Design, 1, 289-295. Goldberg, L. R. (1999). A broad-bandwidth, public-domain, personality inventory measuring the lower-level facets of serveral five-factor models. In Personality psychology in Europe (Vol. 7, pp. 7-28). Tilburg, The Netherlands: Tilburg University Press. Goldschmidt, G. (2014). Linkography: Unfolding the Design Process. Cambridge, MA: MIT Press. Goldschmidt, G., & Tatsa, D. (2005). How good are good ideas? Correlates of design creativity. Design Studies, 26, 593-611. Goncalo, J. A., Flynn, F. J., & Kim, S. H. (2010). Are two narcissists better than one? The link between narcissism, perceived creativity, and creative performance. Personality and Social Psychology Bulletin, 36, 1484-1495. Goncalo, J. A., & Staw, B. M. (2006). Individualism-collectivism and group creativity. Organizational behavior and human decision processes, 100, 96-109. Guilford, J. P. (1950). Creativity. American Psychologist, 5, 444-454. Guilford, J. P. (1957). Creative Abilities in the arts. Psychological Review, 64, 110. Gutierrez, E. (2009). What's a good idea? Understanding evaluation and selection of new product ideas. Paper Presented at International Conference on Engineering Design, Palo Alto, CA, August 24-27 (121-132). Hair, J. F., Anderson, R. E., Tatham, R. L., & Black, W. C. (2005). Multivariate Data Analysis. Upper Saddle Ricer, NJ: Prentice Hall.

130

Hambali, A., Supuan, S. M., Ismail, N., & Nukman, Y. (2009). Application of analytical hierarchy process in the design concept selection of automotive composite bumper beam during the conceptual design stage. Scientific Research and Essays, 4, 198-211. Hammond, J. S., Keeney, R. L., & Raiffa, H. (1998). The hidden traps in decision making. Harvard Business Review, 76, 47-58. Han, S. D., Boyle, P. A., Arfanakis, K., Fleischman, D. A., Yu, L., Edmonds, E. C., & Bennet, D. A. (2012). Neural intrinsic connectivity networks associated with risk aversion in old age. Behavioral Brain Research, 227, 233-240. Hargadon, A. (2003). How Breakthroughs Happen. Boston, MA: Harvard Business School Press. Harrison, D. A., & Klein, K. J. (2007). What's the difference? Diversity constructs as separation, variety, or disparity in organizations. Academy of Management Review, 32, 1199-1228. Hatcher, L., & Stepanski, E. (1994). A step-by-step approach to using the SAS system for univariate and multivariate statistics, 1994. SAS Institute Inc, Cary (NC). Hazelrigg, G. A. (2010). Letter to the Editor re “The Pugh controlled convergence method: model- based evaluation and implications for design theory”. Research in Engineering Design, 21, 143-144. Heath, C., & Tversky, A. (1991). Preferences and Beliefs: Ambiguity and Competence in Choice Under Uncertainty. Journal of Risk and Uncertainty, 2, 5-35. Heath, T. (1993). Social Aspects of Creativity and Their Impact on Creativity Modelling. Hillsdale, NJ: Erlbaum. Herring, S. R., Chang, C.-C., Krantzler, J., Bailey, B. P., Greenberg, S., Hudson, S., Hinkley, K., RingelMorris, M., & Olsen, D. (2009). Getting Inspired! Understanding How and Why Examples are Used in Creative Design Practice. Paper Presented at CHI Conference on Human Factors in Computing Systems, Boston, MA, April 4-9 (87-96). Hoff, E., Carlsson, I., & Smith, G. (2012). Handbook of Organizational Creativity. London: Elsevier. Hoffman, L. R., Harburg, E., & Maier, N. R. (1962). Differences and disagreement as factors in creative group problem solving. The Journal of Abnormal and Social Psychology, 64, 206. Hoffman, L. R., & Maier, N. (1961). Quality and acceptance of problem solutions by members of homogeneous and heterogeneous groups. Journal of Abnormal and Social Psychology, 62, 401-407. House, T. W. (2009). President Obama Launches "Educate to Innovate" Campaign for Excellence in Science, Technology, Engineering & Math (Stem) Education. In T. W. House (Ed.). Washington, DC: Office of the Press Secretary. Howard, T. J., Culley, S. J., & Dekoninck, E. (2008). Describing the Creative Design Process by the Integration of Engineering Design and Congitive Psychology Literature. Design Studies, 29, 160-180. Hsiung, C. (2012). The Effectiveness of Cooperative Learning. Journal of Engineering Education, 101, 119-137. Huang, H.-Z., Liu, Y., Li, Y., Xue, L., & Wang, Z. (2013). New evaluation methods for conceptual design selection using computational intelligence techniques. Journal of Mechanical Science and Technology, 27, 733-746. Hutchison, M. A., Follman, D. K., Sumpter, M., & Bodner, G. M. (2006). Factors Influencing the Self-Efficacy Beliefs of First-Year Engineering Students. Journal of Engineering Education, 95, 39-47. Ismail, M. (2005). Creative climate and earning organization factors: their contribution towards innovation. Leadership and Organization Development Journal, 26, 639-654.

131

Jacobs, J. F., van de Poel, I., & Osseweijer, P. (2014). Clarifying the debate on selection methods for engineering: Arrow’s impossibility theorem, design performances, and information basis. Research in Engineering Design, 25, 3-10. Johnson, J. (2014). Measuring thirty facets of the Five Factor Model with a 120-item public domain inventory: Development of the IPIP-NEO-120. Journal of Research in Personality, 51, 78- 89. Joshi, A. (2014). By Whom and When Is Women’s Expertise Recognized? The Interactive Effects of Gender and Education in Science and Engineering Teams. Administrative Science Quarterly, 59, 1-38. Kahneman, D., Knetsch, J. L., & Thaler, R. H. (1990). Experimental Tests of the Endowment Effect and the Coase Theorem. Journal of Political Economy, 98, 1325-1348. Kahneman, D., Knetsch, J. L., & Thaler, R. H. (1991). Anomalies: The Endowment Effect, Lost Aversion, and Status Quo Bias. Journal of Economic Perspectives, 5, 193-206. Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica: Journal of the Econometric Society, 47, 263-291. Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica: Journal of the Econometric Society, 263-291. Kan, J. W. T., & Gero, J. S. (2008). Acquiring information from linkography in protocol studies of designing. Design Studies, 29, 315-337. Kazerounian, K., & Foley, S. (2007). Barriers to creativity in engineering education: A study of instructors and student perceptions. Journal of Mechanical Design, 129, 761-768. Kazerounian, K., & Foley, S. (2007). Barriers to creativity in engineering education: A study of instructors and students perceptions. Journal of Mechanical Design, 129, 761-768. Kichuk, S., & Wiesner, W. (1998). The Big Five Personality Factors and Team Performance: Implications for Selecting Successful Product Design Teams. Journal of Engineering and Technology Management, 14, 195-221. Kijkuit, B., & van der Ende, J. (2007). The organizational life of an idea: Integrating social network, creativity and decision-making perspectives. Journal of management studies, 44, 863-882. King, A. M., & Sivaloganathan, S. (1999). Development of a Methodology for Concept Selection in Flexible Design Strategies. Journal of Engineering Design, 10, 329-349. doi:10.1080/095448299261236. Kitayama, S., & Karasawa, M. (1997). Implicit self-esteem in Japan: Name-letters and birthday numbers. Personality and Social Psychology Bulletin, 23, 736-742. Kleiman, P. (2008). Towards transformation: conceptions of creativity in higher education. Innovations in Education and Training International, 45, 209-217. Klein, C., DeRouin, R., & Salas, E. (2006). Uncovering workplace interpersonal skills: A review, framework, and research agenda. International Review of Industrial and Organizational Psychology, 21, 79-126. Kling, K. C., Hyde, J. S., Showers, C. J., & Buswell, B. N. (1999). Gender difference in self-esteem: A meta-analysis. Psychological Bulletin, 125, 470-500. Kremer, G. E., Schmidt, L. C., & Hernandez, N. (2011). An investigation on the impact of the design problem in ideation effectiveness research. Paper Presented at American Society for Engineering Education Conference, Vancouver, B.C., June 26-29 (AC 2011-1356). Kruger, C., & Cross, N. (2006). Solution dtriven versus problem driven design: Strategies and outcomes. Design Studies, 27, 527-548. Kruglanski, A. W., & Webster, D. M. (1996). Motivated closing of the mind: "Seizing" and "freezing". Psychological Review, 103, 263-283.

132

Kudrowitz, B. M., & Wallace, D. (2012). Assessing the quality of ideas from prolific, early-stage product ideation. Journal of Engineering Design, 24, 120-139. doi:10.1080/09544828.2012.676633. Kudrowitz, B. M., & Wallace, D. (2013). Assessing the quality of ideas from prolific, early-stage product ideation. Journal of Engineering Design, 24, 120-139. Kulkarni, C., Dow, S. P., & Klemmer, S. R. (2012). Early and Repeated Exposure to Examples Improves Creative Work. Cognitive Science. Kurdrowitz, B., & Dippo, C. (2013). Getting to the novel ideas: exploring the altenative uses test of divergent thinking. Paper Presented at ASME Design Engineering Technical Conferences, Portland, OR, August 4-7 (10.1115/DETC2013-13262). Lauri, S., & Kyngas, H. (2005). Developing nursing theories. Dark Oy, Vantaa: Werner Söderström. Lenney, E. (1977). Women's self-confidence in achievement settings. Psychological Bulletin, 84, 1-13. Li, Y., Wang, J., Li, X., & Zhao, W. (2007). Design Creativity in Product Innovation. International Journal of Manufacturing Technology, 33, 213-222. Linnerud, B., & Mocko, G. (2013). Factors that Effect Motivation and Performance on Innovative Design Projects. Paper Presented at ASME Design Engineering Technical Conferences, Portland, OR, August 4-7. Linsey, J. S., Clauss, E. F., Kurtoglu, T., Murphy, J. T., Wood, K. L., & Markman, A. B. (2011). An Experimental Study of Group Idea Generation Techniques: Understanding the Roles of Idea Representation and Viewing Methods. Journal of Mechanical Design, 133. doi:031008-1. Litzinger, T. A., Lattuca, L. R., Hadgraft, R. G., & Newsletter, W. C. (2011). Engineering Education and the Development of Expertise. Journal of Engineering Education, 100, 123- 150. López-Mesa, B., & Bylund, N. (2011). A study of the use of concept selection methods from inside a company. Research in Engineering Design, 22, 7-27. Lovallo, D. P., & Sibony, O. (2010). The case for behavioral stategy. McKinsey Quarterly, 2, 30- 43. Macht, G. A., & Nembhard, D. A. (2015). Measures and models of personality and their effects on communication and team performance. International Journal of Industrial Ergonomics, 49, 78-89. Mann, R. (1959). A review of the relationships between personality and performance in small groups. Psychological Bulletin, 56, 241-270. Marsh, E. R., Slocum, A. H., & Otto, K. N. (1993). Hierarchical decision making in machine design. In: MIT Precision Engineering Research Center. Martin, D. J., Dakers, J., Duvernet, L., Kipperman, D., Kumar, K., & Siu, K. W. M. (2003). In search of a sustainable future: An international overview of the contribution from design and technology education. The Journal of Design and Technology Education, 8, 1370149. Mayring, P. (2004). Qualitative content analysis. In U. Flick, E. Kardoff & I. Steinke (Eds.), A companion to qualitative research (pp. 266-269). Thousand Oaks, CA: Sage Publications. McCrae, R. (1987). Creativity, divergent thinking, and openness to experience. Journal of Personality and Social Psychology, 52, 1258-1275. McGrath, J. E. (1998). A view of group composition through a group-theoretic lens. In Research on managing groups and teams (pp. 255-272). Greenwich, CT: JAI. McMurray, A., Scott, D., & Wayne, P. (2004). Research: A commonsense approach. Melbourne, Australia: Cengage Learning.

133

Medin, D. L., Goldstone, R. L., & Markman, A. B. (1995). Comparison and choice: relations between similarity processes and decision processes. Psychonomic Bulletin & Review, 2, 1-19. Mednick, S. A. (1962). The associative basis of the creative process. Psychological Review, 69, 220-232. Meehan, A. M., & Overton, W. F. (1986). Gender differences in expectancies for success and performance on Piagetian spatial tasks. Merril-Palmer Quarterly, 32, 427-441. Miller, G. A. (1956). The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information. The Psychological Review, 63, 81-97. Mohammed, S., & Angell, L. C. (2003). Personality heterogeneity in teams: Which differences make a difference for team performance. Small group research, 34, 651-677. Moore, E., & Eckel, C. (2003). Measuring Ambiguity Aversion. Paper Presented at Economic Science Association Meetings, Barcelona, June 21-24 (n.pag.). Moos, R. H., & Speisman, J. C. (1962). Group compativiltiy and productivity. Journal of Abnormal and Social Psychology, 65, 190-196. Moscovici, S. (1976). Social influence and social change. In: London UK: Academic Press. Mounarath, R., Lovallo, D., & Dong, A. (2011). Choosing innovation: How reasoning affects decision makers. Paper Presented at International Conference on Engineering Design, Copenhagen, Denmark, August 15-18 (1-10). Mueller, J. S., Melwani, S., & Goncalo, J. A. (2011). The Bias Against Creativity: Why People Desire But Reject Creative Ideas. Psychological science, 2011, 0956797611421018. Mumford, M. D. (2003). Where have we been, where are we going? Taking stock in creativity research. Creativity Research Journal, 15, 107-120. Mumford, M. D. (2012). Handbook of Organizational Creativity. San Diego, CA: Academic Press. Nelson, B. A., & Wilson, J. O. (2009). Redefining metrics for measuring ideation effectiveness. Design Studies, 30, 737-743. Nelson BA, Y. J. (2009). Refined metrics for measuring ideation effectiveness. Design Studies, 30, 737-743. Nicholl, B., & McLellan, R. (2008). We’re all in this game whether we like it or not to get a number of As to Cs: Design and technology teachers’ struggles to implement creativity and performativity policies. British Educational Research Journal, 34, 585-600. Nicholson, N., Soane, E., Fenton-O'Creevy, M., & Willman, P. (2005). Personality and Domain- Specific Risk Taking. Journal of Risk Research, 8, 157-176. Nijstad, B. A. (2002). Cognitive stimulation and interference in groups: Exposure effects in an idea generation task. Journal of Experimental Social Psychology, 38, 535-544. Nikander, J. B., Liikkanen, L. A., & Laakso, M. (2014). The preference effect in design concept evaluation. Design Studies, 35, 473-499. doi:http://dx.doi.org/10.1016/j.destud.2014.02.006. Norman, K. (1960). Some organizational factors affecting creativity. Engineering Management, 1, 24-30. Oak, A. (2010). What can talk tell us about design? Analyzing conversation to understand practice. Design Studies, 32, 211-234. Ogot, M., & Okudan-Kremer, G. E. (2006). Engineering Design: A Practical Guide. Bloomington, IN: Trafford Publishing. Okudan, G. E., & Tauhid, S. (2008). Concept Selection Methods – A Literature Review from 1980 to 2008. International Journal of Design Engineering, 1, 243-277. Okudan, G. E., & Tauhid, S. (2008). Concept selection methods–a literature review from 1980 to 2008. International Journal of Design Engineering, 1, 243-277.

134

Onarheim, B., & Christensen, B. T. (2012). Distributed idea screening in stage-gate development processes. Journal of Engineering Design, 23, 660-673. Osborn, A. (1957). Applied Imagination. New York, NY: Scribner. Pahl, G., & Beitz, W. (1984). Engineering Design. London: The Design Council. Parnes, S. J. (1961). Effects of extended effort in creative problem solving. Journal of Educational Psychology, 52, 117-122. Paulus, P. B. (2000). Groups, Teams, and Creativity: The Creative Potential of Idea-Generating Groups. Applied Psychology: An International Review, 49, 237-262. Pearsall, M. J., Ellis, A. P. J., & Evans, J. M. (2008). Unlocking the effects of gender faultlines on team creativity: Is activation key? Journal of applied psychology, 93, 225-234. Pelham, B. W., Mirenberg, M. C., & Jones, J. T. (2001). Why Susie sells seashells by the seashore: Implicit egotism and major life decsiions. Attitudes and Social Cognition, 82, 469-487. Perry-Smith, J. E. (2006). Social yet creative: the role of social relationships in facilitating individual creativity. Academy of Management Journal, 49, 85-101. Phelan, S., & Young, A. M. (2003). Understanding creativity in the workplace: An examination of individual styles and training in relation to creative confidence and creative self-leadership. Journal of Creative Behavior, 37, 266-281. Pierce, J. L., Kostova, T., & Dirks, K. T. (2003). The state of psychological ownership: Integrating and extending a century of research. Review of General Psychology, 8, 84-107. Pugh, S. (1991). Total design: integrated methods for successful product engineering. Workingham: Addison-Wesley. Pugh, S. (1996). Creating Innovative Products Using Total Design. Boston, MA: Addison-Wesley Longman Publishing Co., Inc. QSR. (2012). NVivo Qualitative Data Analysis Software. QSR International Pty Ltd, Version 10. Reilly, R. R., Lynn, G. S., & Aronson, Z. H. (2001). The role of personality in new product development team performance. Journal of Engineering and Technology Management, 19, 39-58. Reiter-Palmon, R. (2009). Problem identification and construction: What do we know, what is the future? Psychology of Aesthetics, Creativity, and the Arts, 3, 43-47. Rhodes, M. (1961). An analysis of creativity. Phi Delta Kappan, 305-310. Richards, L. G. (1998). Stimulating Creativity: Teaching Engineers to be Innovators. Paper Presented at Frontiers in Education Conference, Tempe, AZ, Nov 4-7 (1034-1039). Ridgeway, C. L., & Smith-Lovin, L. (1999). The gender system and interaction. Annual Review of Sociology, 25, 191-216. Rietzchel, E. F., Nijstad, B. A., & Stroebe, W. (2006). Productivity is not enough: a comparison of interactive and nominal groups in idea generation and selection. Journal of Experimental Social Psychology, 42, 244-251. Rietzschel, E., BA Nijstad, & W. Stroebe. (2010). The selection of creative ideas after individual idea generation: choosing between creativity and impact. British Journal of Psychology, 101, 47-68. Rietzschel, E., Nijstad, B., & Stroebe, W. (2010). The selection of creative ideas after individual idea generation: choosing between creativity and impact. British Journal of Psychology, 101, 47-68. Rietzschel, E. F., Nijstad, B. A., & Stroebe, W. (2010). The selection of creative ideas after individual idea generation: choosing between creativity and impact. British Journal of Psychology, 101, 47-68. Roese, N. J., & Olson, J. M. (1994). Attitute importance as a function of repeated attitute expression. Journal of Experimental Social Psychology, 30, 39-51.

135

Ross, M., & Sicoly, F. (1979). Egocentric biases in availability and attribution. Journal of Personality and Social Psychology, 3, 322-336. Rowe, P. G. (1987). Design Thinking. Cambridge, MA: MIT Press. Rubenson, D. L., & Runco, M. A. (1995). The psychoeconomic view of creative work in groups and organizations. Creativity and Innovation Management, 4, 232-241. Ryan, P. (2005). Improv. Wisdom. New York, NY: Bell Tower. Saaty, T. L. (2008). Decision making with the analytic hierarchy process. International Journal of Services Sciences, 83-98. Salonen, M., & Perttula, M. (2005). Utilization of Concept Selection Methods: A Survey of Finnish Industry. Paper Presented at ASME Design Engineering Technical Conferences, Long Beach, California, September 24-28 (527-535). Sarbacker, S. D., & Ishii, K. (1997). A framework for evaluating risk in innovative product development. Paper Presented at Design Engineering Technical Conferences, Sacramento, CA, September, 14-17 (1-10). Shah, J., Kulkarni, S., & Vargas-Hernandez, N. (2000). Evaluation of idea generation methods for conceptual design: effectiveness metrics and design of experiments. Journal of Mechanical Design, 122, 377-384. Shah, J., & Vargas-Hernandez, N. (2003). Metrics for Measuring Ideation Effectiveness. Design Studies, 24, 111-124. Shah, J. J., Smith, S. M., & Woodward, J. (2009). Development of standardized tests for design skills. Paper Presented at International conference on engineering design (ICED09), Stanford, CA, August, 24-27 (269-280). Shah, J. J., Vargas-Hernandez, N., & Smith, S. M. (2003). Metrics for Measuring Ideation Effectiveness. Design Studies, 24, 111-134. Silvia, P. J., Kaufman, J. C., Reiter-Palmon, R., & Wigert, B. (2011). Cantankerous creativity: Honesty-Humility, Agreeableness and the HEXACO structure of creative achievement. In Psychology Faculty Publications: Department of Psychology st DigitalCommons@UNO. Simpson, T., & Thevenot, H. (2007). Using Product Dissection to Integrate Product Family Design Research into the Classroom and Improve Students' Understanding of Platform Commonality. International Journal of Engineering Education, 23, 120-130. Simpson, T. W., Lewis, K. E., Stone, R. B., & Regli, W. C. (2007). Using Cyberinfrastructure to Enhance Product Dissection in the Classroom. Paper Presented at Industrial Engineering Research Conference, Nashville, TN, May 19-23 (http://hdl.handle.net/10355/32582). Sitkin, S. B., & Pablo, A. L. (1992). Reconceptualizing the determinants of risk behavior. Academy of Management Review, 17, 9-38. Siu, K. W. M. (2009). Review on the development of design education in Hong Kong: The need to nurthre the problem finding capability of design students. Educational Research Journal, 23, 179-202. Sivaloganathan, S., & King, A. M. (1999). Development of a Methodology for Concept Selection in Flexible Design Strategies. Journal of Engineering Design, 10, 329-349. Slaughter, E. S. (1998). Models for Construction Innovation. Journal of Construction Engineering and Management, 124, 226-231. Smith, G. J. (1998). Idea-generation technique: A formulary of active ingredients. Journal of Creative Behavior, 107-133. Somech, A., & Drach-Zahavy, A. (2011). Translating team creativity to innovation implementation: The role of team composition and climate for innovation. Journal of Management, 39, 684-708. Sonnenwald, D. H. (1996). Communication roles that support collaboration during the design process. Design Studies, 17, 277-301.

136

Sorrentino, R., & Roney, C. J. R. (2000). The Uncertain Mind: Individual Differences in Facing the Unknown (Vol. 1). Hove, UK: Psychology Press. Stafford, L., Ng, W., Moore, R., & Bard, K. (2010). Bolder, Happier, Smarter: The Role of Extraversion in Positive Mood and Cognition. Personality and Individual Differences, 48, 827-832. Staw, B. M. (1995). Why no one really wants creativity. Thousand Oaks, CA: Sage Publications. Staw, B. M. (1995). Why no one really wants creativity. In Creative Action in Organizations: Ivory Tower Visions and Real World Voices. Thousand Oaks, CA: Sage Publications. Steel, G., T, R., & J, F. (2012). Personality, nations, and innovatio: Relationships between personality traits and national innovation scores. Cross-Cultural Research: The Journal of Comparative Social Science, 46, 3-30. Steel, G., T, R., & J, F. (2012). Personality, nations, and innovation: Relationships between personality traits and national innovation scores. Cross-Cultural Research: The Journal of Comparative Social Science, 46, 3-30. Stempfle, J., & Badke-Schaub, P. (2002). Thinking in design teams- an analysis of team communication. Design Studies, 23, 473-496. Sternberg, R. J., & Lubart, T. I. (1991). An Investment Theory of Creativity and its Development. Human Development, 24, 1-31. Sternberg, R. J., O'Hara, L. A., & Lubart, T. I. (1997). Creativity as an Investment. California Management Review, 40, 8-21. Stone, N. J., Moroney, W. F., & Wortham, T. B. (2006). Recommendations for Teaching Team Bahavior to Human Factors/ Ergonomics Students. Paper Presented at Human Factors and Ergonomics Society Annual Meeting, San Francisco, CA, October 16-20 (784-788). Stouffer, W. B., Russel, J. S., & Oliva, M. G. (2004). Making the Strange Familiar: Creativity and the Future of Engineering Education. In Proceedings of the American Society for Engineering Education Annual Conference (pp. 9315-9327). Salt Lake City, UT. Stouffer, W. B., Russel, J. S., & Oliva, M. G. (2004). Making the Strange Familiar: Creativity and the Future of Engineering Education. Paper Presented at American Society for Engineering Education Annual Conference & Exposition, Salt Lake City, UT, June 20-23 (20-23). Sullivan, J. F., Carlson, L. E., & Carlson, D. W. (2001). Developing Aspiring Engineers into Budding Entrepreneurs: An Invention and Innovation Course. Journal of Engineering Education, 90, 571-576. Takai, S., & Ishii, K. (2010). The Maximum Resource Allocation for Uncertainty Reduction in a Decision-analytic Concept Selection. Concurrent Engineering: Research and Application, 18, 19-29. Telenko, C., Sosa, R., & Wood, K. L. (2014). Changing conversations and perceptions: The research and practice of design science. In Impact of Design Research on Practice (Vol. in press): Springer-Verlag. Thaler, R. H., & Johnson, E. J. (1990). Gambling with the house money and trying to break even: the effects of prior outcomes on risky choice. Management science, 36, 643-660. Toh, C., Miele, L., & Miller, S. (2015). Which one should I pick? Concept selection in engineering design industry. Paper Presented at Design Engineering Technical Conferences, Boston, MA, August 2-5, 2015. Toh, C., & Miller, S. (2014). The role of individual risk attitudes on the selection of creative concepts in engineering design. Paper Presented at ASME Design Engineering Technical Conferences, Buffalo, NY, August 17-20. Toh, C. A., & Miller, S. R. (2014). The Impact of Example Modality and Physical Interactions on Design Creativity. Journal of Mechanical Design, 136. doi:10.1115/1.4027639.

137

Toh, C. A., & Miller, S. R. (2016). Choosing Creativity: The Role of Individual Risk and Ambiguity Aversion on Creative Concept Selection in Engineering Design. Research in Engineering Design, January, 1-25. Toh, C. A., & Miller, S. R. (2016). Creativity in Design Teams: The Influence of Personality Traits and risk Attitudes on Creative Concept Selection. Research in Engineering Design, 27, 73- 89. Torrance, E. P. (1999). Torrance Test of Creative Thinking: Norms and Technical manual. Beaconville, IL: Scholastic Testing Services. Ulrich, K. T., Eppinger, S. D., & Goyal, A. (2011). Product design and development. New York, NY: McGraw-Hill. Ulrich, K. T., Eppinger, S. D., & Goyal, A. (2011). Product design and development (Vol. 2): McGraw-Hill. Valicer, W. F., & Fava, J. L. (1998). Effects of variable and subject sampling on factor pattern recovery. Psychological Methods, 3, 231-251. Van De Van, A., & Delbecq, A. L. (1974). The effectiveness of nominal, Delphi, and interacting group decision making processes. The Academy of Management Journal, 17, 605-621. Van Swol, L. M. (2007). Perceived Importance of Information: The Effects of Mentioning Information, Shared Information Bias, Ownership Bias, Reiteration, and Confirmation Bias. Group Processes & Intergroup Relations, 10, 239-256. Wallach, M. A., & Kogan, N. (1965). A new look at the creativity-intelligence distinction1. Journal of Personality, 33, 348-369. Weber, E. U. (2010). Risk attitude and preference. WIley Interdisciplinary Reviews; Cognitive Science, 1, 263-290. Weber, E. U., Blais, A.-R., & Betz, N. E. (2002). A domain-specific risk-attitude scale: measuring risk perceptions and risk behaviors. Journal of Behavioral Decision Making, 15, 263-290. doi:10.1002/bdm.414. Weck, O., Eckert, C., & Clarkson, J. (2007). A classification of uncertainty for early product and system design. Paper Presented at International Conference on Enineering Design, ICED, Paris, France, August, 28-31 (1-12). Wells, J. D., Campbell, D. E., Valacich, J. S., & Featherman, M. (2010). The effect of perceived novelty on the adoption of information technology innovations: a risk/reward perspective. Decision Sciences, 41, 813-843. Westby, E. L., & Dawson, V. L. (1995). Creativity: Asset of burden in the classroom? Creativity Research Journal, 8, 1-10. Whitson, J., & Galinksy, A. (2008). Lacking control increasing illusory pattern perception. science, 322, 115-117. Wilde, D. J. (1997). Using student preferences to guide design team composition. Paper Presented at Design Engineering Technical Conferences, Sacramento, CA, September 14-17 (1-6). Wiltschnig, S., Christensen, B. T., & Ball, L. J. (2013). Collaborative problem–solution co- evolution in creative design. Design Studies, 34, 515-542. Wittenbaum, G. M., Hubbell, A. P., & Zuckerman, C. (1999). Mutual enhancement: Toward an understanding of the collective preference for shared information. Journal of Personality and Social Psychology, 77, 967-978. Woodman, R. W., Sawyer, J. E., & Griffin, R. W. (1993). Toward a theory of organizational creativity. Academy of Management Review, 18, 293-321. Xu, H., & Brucks, M. (2011). Are Neurotics Really More Creative? Neuroticism's Interaction with Mortality Salience in Determining Creative Interest. Basic and Applied Social Psychology, 33, 88-99.

138

Yang, M. C. (2009). Observations on concept generation and sketching in engineering design. Research in Engineering Design, 20, 1-11. Yang, M. C., & Epstein, D. J. (2005). A study of prototypes, design activity, and design outcomes. Design Studies, 26, 649-669. Zenasni, F., Besancon, M., & Lubart, T. (2008). Creativity and Tolerance of Ambiguity: An Empirical Study. The Journal of Creative Behavior, 42, 61-73. Zuckerman, M., & Kuhlman, D. M. (2000). Personality and Risk-Taking: Common Biosocial Factors. Journal of Personality, 68, 999-1029.

139

APPENDIX A: INSTRUCTIONS FOR DESIGN TASKS

BRAINSTORMING INSTRUCTIONS FOR MILK FROTHER DESIGN TASK

Upper management has put your team in charge of developing a concept for a new innovative product that froths milk in a short amount of time. Frothed milk is a pourable, virtually liquid foam that tastes rich and sweet. It is an ingredient in many coffee beverages, especially espresso-based coffee drinks (Lattes, Cappuccinos, Mochas). Frothed milk is made by incorporating very small air bubbles throughout the entire body of the milk through some form of vigorous motion. As such, devices that froth milk can also be used in a number of other applications, such as for whipping cream, blending drinks, emulsifying salad dressing, and many others. This design your team develops should be able to be used by the consumer with minimal instruction. It will be up to the board of directors to determine if your project will be carried on into production.

Once again, the goal is to develop concepts for a new, innovative product that can froth milk in a short amount of time. This product should be able to be used by the consumer with minimal instruction.

Sketch your ideas in the space provided in the idea generation sheets. As the goal of this design task is not to produce a final solution to the design problem but to brainstorm ideas that could lead to a new solution, feel free to explore the solution space and focus on both the form and function of the design in order to develop innovative concepts. In other words, generate as many ideas as possible- do not focus on the feasibility or detail of your ideas. You may include words or phrases that help clarify your sketch so that your concept can be understood easily by anyone.

For clarity, please use the provided pen to generate your concepts (ie: do not use pencil). Your participant number is included on each of the provided idea generation sheets. Generate one idea per sheet and label the idea number at the top of the sheet.

140

BRAINSTORMING INSTRUCTIONS FOR UTI TEST STRIP DESIGN TASK Penn State’s HESE (Humanitarian Engineering and Social Entrepreneurship) program has developed affordable test strips for the detection of urinary tract infections (UTIs). Worldwide, UTIs are one of the most commonly contracted bacterial infections among pregnant women. If detected and treated early, these infections are not typically life-threatening but are associated with painful and more frequent urination. Left untreated, however, these infections have the potential to become much more serious, even life-threatening.1 The strips developed by the HESE team test for three UTI markers measured within a urine sample: (1) increased pH level; (2) presence of E. Coli catalase; and (3) presence of nitrites. The test strips are printed using a standard inkjet printer, providing the opportunity to print the test strips in country, reducing the associated cost and increasing availability. The typical test strip is small; with dimensions of 9 x 100 mm, 100 strips fit on a http://ukweliteststrips.weebly.com/our- standard A4 sheet of paper. Distribution of the strips to technology.html the targeted customers – pregnant women – will be accomplished through community health workers that have been trained to assess the test and provide a diagnosis. Early detection and subsequent treatment of the UTI is expected to reduce morbidity and cost. Currently, the major design hurdle to overcome is the mechanism by which the test strip is exposed to the urine sample (e.g., immersion, placement of the strip within the urine stream, etc.). Therefore, a reliable solution with significant potential for widespread adoption is sought. Also, a new test strip is currently under development to test for sugar in the urine – one of the signs of diabetes. With current global incidence numbers at over 380 million individuals and annual worldwide expenditure of nearly $550 billion (and both predicted to grow), diabetes is a health problem of epidemic proportions around the world.2 Therefore, a successful solution for the UTI test strip that is adaptable for glucose could have even broader impact and benefit. Another central tenet is for the Test Strip project is the concept of Cradle to Cradle design - a biomimetic approach to the design of products and systems. It is a holistic economic, industrial and social framework that seeks to create systems that are efficient and essentially waste free. The model is not limited to industrial design and manufacturing; it can be applied to many aspects of human civilization such as urban environments, buildings, economics and social systems.

The end goal of this project is a simple, inexpensive, low-waste, and durable system that allows for introduction of a urine sample onto the test strip. The implementation of this new component with the currently manufactured test strip should be accomplished in country with locally-available materials (or those easily and inexpensively imported).

1 S. August and M. De Rosa, “Evaluation of the Prevalence of Urinary Tract Infection in Rural Panamanian Women,” PLOS ONE 7(10), 2012. 2 International Diabetes Federation. IDF Diabetes Atlas, 6th edn. Brussels, Belgium: International Diabetes Federation, 2013.

141

BRAINSTORMING INSTRUCTIONS FOR GREENHOUSE GRID DESIGN TASK

Over the last 4 years, Penn State’s HESE (Humanitarian Engineering and Social Entrepreneurship) program has been refining the design for a low-cost greenhouse for small-scale famers which enables them to move from subsistence to sustainability. Greenhouses can allow farmers to grow vegetables and fruits year-round, increase their yields and improve their livelihoods while reducing spoilage and providing food security. A Penn State greenhouse team has collaborated with Kenyan, Rwandan and Tanzanian entities to design, prototype, and field-test affordable greenhouses designed for small agro-enterprises and sustenance farmers. Last year, the Penn State team contractually licensed our greenhouse design solution to a for-profit company called Mavuuno Greenhouses Limited - http://www.mavuunogreenhouses.com. Mavuuno manufactures Greenhouse Kits for the East African market, which are sold through a network of distributors and construction agents who assemble the Greenhouse Kits on farms and train the farmers on startup and maintenance regimens. In addition, HESE has a licensing agreement with The Greenhouse Center, another for-profit company based in Cameroon for the West African Market. Finally, with Photos provided by HESE (2014). USAID support, HESE is working with World Hope International to jumpstart GRO Greenhouses in Mozambique and Sierra Leone over the next year.

The Mavuuno greenhouse design has been well researched and the selection of materials optimized for location, minimal cost, and durability. To set up the greenhouse, the carpenter (construction agent) goes to a site with the Greenhouse Kit and hires 1-2 local laborers at about $3 - $6 per day (8 hours). These local laborers are typically young men and readily available because of the high unemployment rates: youth unemployment is about 70% in Sierra Leone.

Some critical steps that must be done before construction of the frame:

1. Clear and level a 7 meter x 7 meter area for the 6 meter x 6 meter greenhouse. 2. Locate one corner of the greenhouse, and lay out the 6 meter x 6 meter grid for the frame posts in a square pattern. 3. Dig the post holes, construct the frame and attach the glazing material to the frame.

Parts of Step 1 and 2 form this DEM project challenge. With only simple tools such as rope or wire, a level, and a measuring tape, 1) define an efficient method for measuring how level the ground is, and 2) define the process so a 6 meter x 6 meter square area can be marked with 49 frame post locations which are square. Any one post can be no more than 1 centimeter off. The goal is to completely mark the grid (start to finish) in 10 minutes or less. Other factors: assume the person laying out the grid cannot read or write, and the wood for the greenhouse frame will be warped. Design of very simple tool(s) to help with measuring ground “levelness” and do the grid layout process is encouraged. However, any new device must not be too heavy and should be ruggedized for harsh environments. The budget for any new device is $10. Available materials are nylon string, wood and metal bars.

142

BRAINSTORMING INSTRUCTIONS FOR REDUCING PEDESTRIAN ACCIDENTS DESIGN TASK

There has been an increase in student accidents on campus in recent years from student’s texting and/ or talking on mobile devices or listening to music using earphones while walking around campus. While using these devices, students become distracted, and can trip, fall or even collide into obstacles. In fact, the number of pedestrian ER visits for injuries related to cell phones tripled between 2004 and 2010. There are reports of concussions, sprained ankles, broken appendages and even fatalities from these accidents. These numbers do not include the countless number of unreported incidents involving walking into something (i.e. a parked car) without an ER visit. This increase in accidents has been substantial on college campuses because of the number of students on campus and the increased usage of mobile devices (listening to music, texting, and talking) all of which are distracting.

Your task is to develop concepts for a new, innovative product or system that will reduce pedestrian accident rates due to distraction from mobile devices.

For clarity, please use the provided pen to generate your concepts (ie: do not use pencil). Include your participant number on each of the provided idea generation sheets. Generate one idea per sheet and label the idea number at the top of the sheet.

143

INDIVIDUAL CONCEPT ASSESSMENT INSTRUCTIONS

During this activity, you will review and assess the concepts that you and your team have generated to address the design goal. Once again, the goal of this design problem is to develop concepts for a new, innovative, product that can froth milk in a short amount of time. Your task is to individually assess all of the generated concepts for the extent to which they address the design goal effectively, using the following instructions (illustrated in the diagram below):

1. Shuffle all of the concepts that you have generated in random order. Pass all of the designs you have generated to the team member sitting to your right.

2. After receiving the concepts that were passed to you from the team member sitting to your left, rate each concept in the order that you received them using the rating table provided to you in this booklet. For each concept that you rate, record the corresponding participant’s number, idea number, and a brief description of the concept (e.g., “Double frothing attachments”). You will be given 5 minutes to interpret the designs that you receive without conversing with your team members. For your reference, definitions of the rating scale items have been provided below:

Consider: Concepts in this category are the concepts that will most likely satisfy the design goals, you want to prototype and test these ideas immediately. It may be the entire design that you want to develop, or only 1 or 2 specific elements of the design that you think are valuable for prototyping or testing.

3. Repeat step 2, passing designs that are already rated to your right, and rate designs that are passed to you from the left. You will be given 5 minutes to rate each set of design ideas.

144

TEAM CONCEPT SELECTION INSTRUCTIONS

During this activity, you will once again review and assess the concepts that you and your team have generated to address the design goal in a team setting. Once again, the goal of this design problem is to develop concepts for a new, innovative, product that can froth milk in a short amount of time. Your task is to assess all of the generated concepts for the extent to which they address the design goal effectively in your design teams, using the following instructions:

1. Collect all concepts that your team has generated and shuffle them in random order. As a team, discuss which concepts should be ‘Considered’ and classified as ‘Do Not Consider’. Categorize all the concepts your team has developed by placing them on the table with the corresponding category labels. For your reference, the category definitions have once again been provided below:

Consider: Concepts in this category are the concepts that will most likely satisfy the design goals, Your team wants to prototype and test these ideas immediately. It may be the entire design that your team wants to develop, or only 1 or 2 specific elements of the design that you think are valuable for prototyping or testing.

Do Not Consider: Concepts in this category have little to no likelihood of satisfying the design goals and your team finds minimal value in these ideas. These designs will not be prototyped or tested in the later stages of design because there are no elements in these concepts that your team would consider implementing in future designs.

2. After all concepts have been categorized, rank all concepts in the ‘Consider’ category only. As a team, come to a consensus on the rankings of the concepts. Place the Post-it notes on the concepts to rank them, with 1 being the best concept, 2 being second best, and so on.

145

APPENDIX B: DESIGN RATING SURVEYS (DRS)

MILK FROTHER DRS

1. Is the device handheld? o Yes, it’s handheld o No o Not Explicitly Stated

(if not handheld) 2. If the device is NOT handheld, what does it look like?

o it has a stand (for the counter-top) o its goes in or is attached to a cup (includes a handle) o it goes in or is attached to a bowl (does not include a handle) o it goes in or is attached to a pitcher/ blender o It’s attached to a coffee maker-type device o Other, describe:______

(If handheld) 3. Since the device is handheld, what does the handheld surface look like? o It closely resembles the example o It has a different size (longer, shorter, thinner, wider, etc) than the example o It has finger grips o It has an ergonomic grip o It is held differently than example. o It is rounded/ curved. o Other, describe (e.g. ‘gun shape’) : ______o Not Explicitly Stated

(If handheld) 4. What material is the device’s body made of? o Plastic o Metal o Other (describe e.g. ‘gel’): ______o Not Explicitly Stated

5. How is the device powered? o Manually powered (e.g. hand pump) o Electric o Other, describe: ______o Not Explicitly Stated

(if the device is powered by electricity) 6. What is the device’s electrical source? o AC (Plugs into wall or some other source) o Battery(ies), non rechargeable. o Rechargeable

146

o Solar o Other, describe: ______o Not Explicitly Stated

(if powered by batteries) 7. Where are the device’s batteries inserted? o At bottom of device with slide cover like example o At bottom of device with screw cap o At bottom of device with other (describe): ______o Other location, describe: ______o Not Explicitly Stated

(if powered by batteries) 8. How are the batteries connected? o In parallel, like the example o In series o There is only 1 battery. o Other type of connection, describe: ______o Not Explicitly Stated

(if the device is powered by electricity) 9. How is the device turned on? o By toggle switch, like in the example o By push button o By a switch (unspecified type) o By selecting a speed. o Other, describe: ______o NA

(if the device is powered by electricity) 10. Where is the power switch located? o On the side, like in the example o On the side, unlike the example o On top. o Other, describe: ______o Not Explicitly Stated

11. Where is the liquid (milk) stored for frothing? o Outside of the device, like in the example. o Inside of the device. o Other, describe: ______o Not Explicitly Stated

12. Is there a rod in the design? o Yes o No

(If there is a rod in the design) 13. What does the device’s rod look like? o It connects the main body or motor of the device to an attachment, as in the example. o It’s a different size (length or thickness) than the example

147

o It’s made of a different material o There are multiple rods o It’s a different shape o It’s retractable o Other, describe: ______o Not Explicitly Stated

(if there is a rod) 14. Is there an attachment at the end of the rod? o Yes o No

(if there is an attachment at the end of the rod) 15. How does the attachment (at the end of the rod) differ from the original design? o It doesn’t o It’s a different size o There are multiple attachments o It is made of a different material. o It has a different amount of flexibility. o It has a different shape, describe (e.g. metal spokes, beater, propeller, paddle, etc): ______o It is oriented differently on the device o Other, describe: ______o Not Explicitly Stated

16. What method does the device use to froth the milk? • Stirring, like in the example. • Steam • Spinning (a container of milk) • Pumping • Shaking or vibrating the entire body of milk • Bubbles/ air • Microwave/ waves of some type • Chemicals • Heat • Laser • Pressure/ pressurized milk • Vibrations • Magic • Not Explicitly Stated

(If frothed by stirring) 17. What kind of motion does the device use to stir the milk?

o Circular, in 1 direction, like the example. o Circular, in multiple directions o Up and down o Side to Side o Other, describe: ______o Not Explicitly Stated

148

18. Does the concept focus on motor, electrical wirings, or the batteries of the device? o Yes o No

(if the concept focuses on the motor, electrical wirings, or batteries of the device) 19. Since the concept focuses on the motor, electrical wirings, or the batteries of the device, what part does it focus on? o The wires/ connectors o The motor (e.g. changing DC motor, pump) o The motor casing/ cover material (e.g. second interior coating to reduce corrosion) o The batteries o Other, describe: ______

20. What additional features are included in the concept? o Lid o Interchangeable attachments (e.g. whisks) o Design (colors, etc.) o Noise level change o Waterproof o Sensor o Adds flavor o Different speed settings o Other, describe: ______o Not Explicitly Stated

21. Does the device froth milk? o Yes o No

(if the device froths milk) 22. Is the device technically feasible (is it possible to make it)? o Yes o No

(if the device is technically feasible) 23. Is the concept easy to execute (is it easy/plausible to manufacture and implement it)? o Yes, even if it may be slightly more complicated. o No, it is either unreasonable to make, or you would never use it to froth milk.

(if the device froths milk) 24. Is the concept a significant improvement over the original design?

o Yes. o No.

149

GREENHOUSE GRID DRS

1. Does the concept focus on a method for laying out a 6m x 6m grid on the ground or a method for leveling the ground? a. Method for laying out 6m x 6m grid b. Method for leveling the ground c. Both d. Not Explicitly Stated

2. (if focuses on laying out 6m x 6m grid) Does the concept focus on measuring out a grid, or marking the grid on the ground? a. Measuring out a grid b. Marking the grid on the ground

3. (if focuses on measuring out a grid) What approach does the concept use to measure out the 6m x 6m grid? a. Measuring shapes b. Sticks/ rods c. Flexible/ hinged linkages d. String e. Wheels f. Vehicle g. Net h. Projection of grid pattern or measurement on the ground i. Footsteps j. Other: ______k. Not Explicitly Stated

4. (if the concept uses measuring shapes) What does the measuring shape look like? a. Triangular b. Square/ rectangular c. Circular d. Cube e. A cross/ ‘X’ f. Other: ______g. Not Explicitly Stated

5. (if the concept uses measuring shapes) What material is the measuring shape made of? a. Paper b. Plastic c. String d. Wood e. Other: ______f. Not Explicitly Stated

6. (if the concept focuses on marking the grid on the ground) What method does the concept use to mark the ground? a. Flying projectiles b. Rolling cylinders with spikes c. Rolling wheels with spikes d. Spray paint e. Manual hand drill f. Shovel

150

g. Other:______h. Not Explicitly Stated

7. (if the concept uses Projection of grid pattern on the ground) What light source does the concept use to project the grid pattern on the ground? a. Sunlight b. Flashlight c. Electronic projector d. Other:______e. Not Explicitly Stated

8. (if focuses on method for leveling the ground) Which approach does the concept use to level the ground? a. Hanging weight b. Container with liquid c. Conventional liquid bubble level d. Tarp e. Stakes f. Shovel g. Other: ______h. Not explicitly stated 9. 10. (if focuses on method for laying out 6m x 6m grid) Does the concept approach the problem using a single step or multiple steps? a. A single step b. Multiple steps c. Not Explicitly Stated

11. (if focuses on method for laying out 6m x 6m grid) Does the concept rely on human labor or machines to satisfy the design goal? a. Human labor b. Machines c. Not Explicitly Stated

12. What additional features does the concept involve? a. Robots b. Lego pieces c. Other: ______d. Not Explicitly Stated

13. Does the concept provide a method for laying out a 6m x 6m grid on the ground or a method for leveling the ground? a. Yes b. No

14. (if the concept provides a method for laying out a 6m x 6m grid on the ground or a method for leveling the ground) Is the concept technically feasible (is it possible to make it)? a. Yes b. No 15. (if the concept is technically feasible) Is the concept easy to execute (is it easy/ plausible to manufacture and implement it)? a. Yes b. No

151

UTI TEST STRIP DRS

1. How is the urine introduced to the UTI test-strip in the concept? a. The test-strip is dipped into urine b. The test-strip is placed in the urine stream c. The test-strip is attached to/ is part of a toilet paper d. The test-strip is attached to/ is part of an article of clothing e. Other: ______f. Not Explicitly Stated

2. (if the test-strip is dipped into urine) What type of container is used to store the urine for testing? a. Cup b. Cone c. Coconut shell/ husk d. Leaves e. Fruit f. Bucket g. Other: ______h. Not Explicitly Stated

3. (if the test-strip is placed in the urine stream) What does the concept look like? a. It is attached to/ is part of a toilet b. It has a funneling device c. Pregnancy-type test sticks d. It has a test strip attached to a separate object e. Other: ______f. Not Explicitly Stated

4. (if the test strip is attached to a separate object) What is the test strip attached to? a. String b. Stick c. Piece of wood d. Grass e. Leaves f. Cup g. Funnel h. Bag i. Ruler j. Clip k. Fishing hook l. Fly swatter m. Other: ______n. Not Explicitly Stated

5. (if test-strip is attached to/ is part of a toilet) How is the test-strip attached to or integrated into a toilet? a. Rod with test-strip across toilet bowl b. Test-strip at bottom of toilet bowl c. Clip onto the lip of the toilet bowl d. Other: ______e. Not Explicitly Stated

152

6. (if test-strip has a funneling device) What does the concept use to funnel urine? a. Conventional funnel b. Leaves c. Pipes/ tubes d. Inclined plane e. Metal strip f. Other: ______g. Not Explicitly Stated 7. (if the concept is attached to/ is part of an article of clothing) What clothes are used? a. Underwear b. Diaper c. Pants d. Skirts

8. Does the concept have disposable components? a. Yes b. No

9. Does the concept involve any electronic components? a. Yes b. No • 10. What additional features does the concept involve? a. Printing out test ticket b. Other: ______c. Not Explicitly Stated

11. Does the concept provide a method for introducing a urine sample to a test strip? a. Yes b. No

12. (if the concept provides a method for introducing a urine sample to a test strip) Is the concept technically feasible (is it possible to make it)? a. Yes b. No

13. (if the concept is technical feasible) Is the concept easy to execute (is it easy/ plausible to manufacture and implement it)? a. Yes b. No

14. (if the concept provides a method for introducing a urine sample to a test strip) Is the concept a significant improvement over the original design (dipping the test strip into a plastic container)? a. Yes b. No

153

REDUCING PEDESTRIAN ACCIDENTS DRS

1. Does the concept focus on developing/ modifying a device or changing the surrounding environment? a. Developing/ modifying a device b. Changing the surrounding environment

2. (if focuses on developing/ modifying a device) What device does the concept focus on? a. Generic cellphone b. Smartphone c. Tablet d. ‘Phablet’ e. Other: ______f. Not explicitly stated

3. (if focuses on developing/ modifying a device) What form does the concept take? a. Is attached to, or is part of an accessory b. Is attached to, or is part of the handheld device

4. (if it is attached to, or is part of an accessory) What does it look like? a. It is attached to earphones/ headphones b. It is attached to shoes c. It is a pair of glasses d. It is a watch/ wristband e. Other: ______

5. (if it is attached to, or is part of the handheld device) What does it look like? a. It is attached to the back of the device b. It is a software improvement c. It is a microphone d. It is a sensor in the device e. It is a sensor that gets attached to the device f. Other: ______

6. (if focuses on developing/ modifying a device) What method does the device use to reduce pedestrian accident rates? a. Reducing distractions b. Tracking the surroundings and/ or obstacles c. Providing visual feedback d. Providing audio feedback e. Providing haptic or tactile feedback f. Other: ______

7. (if device reduces distractions) How does the device reduce distractions? a. Voice activated commands b. Remote controls or external controls (volume, music control) c. Limits on device usage (locking screen, error messages, volume control, etc…) d. Other: ______

8. (if it is a tracking device) How does the device track the surroundings and/ or obstacles? a. GPS technology b. Lasers

154

c. Generic transponder and receiver d. Other: ______

9. (if device provides visual feedback) How does the device provide visual feedback? a. Lights or indicators b. Transparent backgrounds c. Video cameras d. Mirrors e. Other: ______

10. (if device provides audio feedback) How does the device provide audio feedback? a. Alerts (beeps) b. Amplifying or transmitting ambient sounds (car horn, traffic, etc…) c. Automatic volume control d. Other: ______

11. (if it focuses on changing the surrounding environment) What method does the concept use to reduce pedestrian accident rates? a. Alerting pedestrians to obstacles or vehicles b. Alerting vehicles to distracted pedestrians c. Redirecting pedestrian traffic or behavior d. Redirecting vehicle traffic or behavior e. Other: ______

12. (if it alerts pedestrians to obstacles or vehicles) What does the concept look like? a. Warning signs (on the ground, posts, etc…) b. Audio cues at intersections c. Device that limits mobile device functionality at intersections d. Other: ______

13. (if it redirects pedestrian traffic or behavior) What does the concept look like? a. Separate pedestrian zones for walking b. Concrete pillars c. Moving walkways

14. What other additional features does the concept include? a. ______

15. Does the device reduce pedestrian accidents? a. Yes b. No

(if the device reduces pedestrian accidents) 16. Is the device technically feasible (is it possible to make it)? a. Yes b. No

(if the device is technically feasible) 17. Is the concept easy to execute (is it easy/plausible to manufacture and implement it)? a) Yes, even if it may be slightly more complicated. b) No, it is either unreasonable to make, or you would never use it to reduce pedestrian accidents

155

APPENDIX C: ONLINE SURVEY FOR RISK ATTITUDES AND PERSONALITY TRAITS

RISK AVERSION AND AMBIGUITY AVERSION QUESTIONS The following questions assess an individual's risk aversion level. Answer the following questions regarding hypothetical lottery scenarios by specifying whether you prefer a fixed payoff of a specified value, or a gamble involving a fair coin toss with an uncertain payoff of a specified value.

1. Which do you prefer? m $15 for sure m $20 if you flip heads

2. Which do you prefer? m $15 for sure m $100 if you flip heads

3. Which do you prefer? m $15 for sure m $80 if you flip heads

4. Which do you prefer? m $15 for sure m $220 if you flip heads

5. Which do you prefer? m $15 for sure m $50 if you flip heads

6. Which do you prefer? m $15 for sure m $200 if you flip heads

7. Which do you prefer? m $15 for sure m $180 if you flip heads

8. Which do you prefer? m $15 for sure m $250 if you flip heads

9. Which do you prefer? m $15 for sure m $90 if you flip heads

10. Which do you prefer? m $15 for sure m $70 if you flip heads

156

The following questions assess an individual's ambiguity aversion level. Answer the following questions regarding hypothetical lottery scenarios by specifying whether you prefer a fixed payoff of a specified value, or a gamble of unknown odds with an uncertain payoff of a specified value (i.e., it is not known how likely it is for you to win the gamble, as it could range from not at all likely, to extremely likely).

Which do you prefer? m $15 for sure m $20 if you win the gamble with unknown probability and $0 if you do not.

2. Which do you prefer? m $15 for sure m $100 if you win the gamble with unknown probability and $0 if you do not.

3. Which do you prefer? m $15 for sure m $80 if you win the gamble with unknown probability and $0 if you do not.

4. Which do you prefer? m $15 for sure m $220 if you win the gamble with unknown probability and $0 if you do not.

5. Which do you prefer? m $15 for sure m $50 if you win the gamble with unknown probability and $0 if you do not.

6. Which do you prefer? m $15 for sure m $200 if you win the gamble with unknown probability and $0 if you do not.

7. Which do you prefer? m $15 for sure m $180 if you win the gamble with unknown probability and $0 if you do not.

8. Which do you prefer? m $15 for sure m $250 if you win the gamble with unknown probability and $0 if you do not.

9. Which do you prefer? m $15 for sure m $90 if you win the gamble with unknown probability and $0 if you do not.

10. Which do you prefer? m $15 for sure m $70 if you win the gamble with unknown probability and $0 if you do not.

157

PERSONALITY TRAIT QUESTIONS: INSTRUCTIONS FOR COMPLETING THE IPIP-NEO SHORT FORM The following pages contain phrases describing people's behaviors. Please use the rating scale next to each phrase to describe how accurately each statement describes you. Describe yourself as you generally are now, not as you wish to be in the future. Describe yourself as you honestly see yourself, in relation to other people you know of the same sex as you are, and roughly your same age. So that you can describe yourself in an honest manner, your responses will be kept in absolute confidence. Please read each statement carefully, and then click the circle that corresponds to the accuracy of the statement.

Name: Sex: Female Male Age:

1. Worry about things. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 2. Make friends easily. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 3. Have a vivid imagination. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 4. Trust others. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 5. Complete tasks successfully. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 6. Get angry easily. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 7. Love large parties. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 8. Believe in the importance of art. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 9. Use others for my own ends. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 10. Like to tidy up. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 11. Often feel blue. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 12. Take charge. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 13. Experience my emotions intensely. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 14. Love to help others. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 15. Keep my promises. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 16. Find it difficult to approach others. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 17. Am always busy. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 18. Prefer variety to routine. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 19. Love a good fight. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 20. Work hard. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 21. Go on binges. Very Moderately Neither Accurate Moderately Very

158

Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 22. Love excitement. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 23. Love to read challenging material. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 24. Believe that I am better than others. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 25. Am always prepared. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 26. Panic easily. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 27. Radiate joy. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 28. Tend to vote for liberal political Very Moderately Neither Accurate Moderately Very candidates. Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 29. Sympathize with the homeless. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 30. Jump into things without thinking. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 31. Fear for the worst. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 32. Feel comfortable around people. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 33. Enjoy wild flights of fantasy. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 34. Believe that others have good Very Moderately Neither Accurate Moderately Very intentions. Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 35. Excel in what I do. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 36. Get irritated easily. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 37. Talk to a lot of different people at Very Moderately Neither Accurate Moderately Very parties. Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 38. See beauty in things that others Very Moderately Neither Accurate Moderately Very might not notice. Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 39. Cheat to get ahead. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 40. Often forget to put things back in Very Moderately Neither Accurate Moderately Very their proper place. Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 41. Dislike myself. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 42. Try to lead others. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 43. Feel others' emotions. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 44. Am concerned about others. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 45. Tell the truth. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 46. Am afraid to draw attention to Very Moderately Neither Accurate Moderately Very myself. Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 47. Am always on the go. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 48. Prefer to stick with things that I Very Moderately Neither Accurate Moderately Very know. Inaccurate Inaccurate Nor Inaccurate Accurate Accurate

159

49. Yell at people. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 50. Do more than what's expected of Very Moderately Neither Accurate Moderately Very me. Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 51. Rarely overindulge. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 52. Seek adventure. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 53. Avoid philosophical discussions. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 54. Think highly of myself. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 55. Carry out my plans. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 56. Become overwhelmed by events. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 57. Have a lot of fun. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 58. Believe that there is no absolute Very Moderately Neither Accurate Moderately Very right or wrong. Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 59. Feel sympathy for those who are Very Moderately Neither Accurate Moderately Very worse off than myself. Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 60. Make rash decisions. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate Nor Inaccurate Accurate Accurate 61. Am afraid of many things. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 62. Avoid contacts with others. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 63. Love to daydream. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 64. Trust what people say. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 65. Handle tasks smoothly. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 66. Lose my temper. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 67. Prefer to be alone. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 68. Do not like poetry. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 69. Take advantage of others. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 70. Leave a mess in my room. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 71. Am often down in the dumps. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 72. Take control of things. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 73. Rarely notice my emotional Very Moderately Neither Accurate Moderately Very reactions. Inaccurate Inaccurate nor Inaccurate Accurate Accurate 74. Am indifferent to the feelings of Very Moderately Neither Accurate Moderately Very others. Inaccurate Inaccurate nor Inaccurate Accurate Accurate 75. Break rules. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 76. Only feel comfortable with friends. Very Moderately Neither Accurate Moderately Very

160

Inaccurate Inaccurate nor Inaccurate Accurate Accurate 77. Do a lot in my spare time. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 78. Dislike changes. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 79. Insult people. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 80. Do just enough work to get by. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 81. Easily resist temptations. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 82. Enjoy being reckless. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 83. Have difficulty understanding Very Moderately Neither Accurate Moderately Very abstract ideas. Inaccurate Inaccurate nor Inaccurate Accurate Accurate 84. Have a high opinion of myself. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 85. Waste my time. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 86. Feel that I'm unable to deal with Very Moderately Neither Accurate Moderately Very things. Inaccurate Inaccurate nor Inaccurate Accurate Accurate 87. Love life. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 88. Tend to vote for conservative Very Moderately Neither Accurate Moderately Very political candidates. Inaccurate Inaccurate nor Inaccurate Accurate Accurate 89. Am not interested in other people's Very Moderately Neither Accurate Moderately Very problems. Inaccurate Inaccurate nor Inaccurate Accurate Accurate 90. Rush into things. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 91. Get stressed out easily. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 92. Keep others at a distance. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 93. Like to get lost in thought. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 94. Distrust people. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 95. Know how to get things done. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 96. Am not easily annoyed. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 97. Avoid crowds. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 98. Do not enjoy going to art museums. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 99. Obstruct others' plans. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 100. Leave my belongings around. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 101. Feel comfortable with myself. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 102. Wait for others to lead the way. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 103. Don't understand people who get Very Moderately Neither Accurate Moderately Very emotional. Inaccurate Inaccurate nor Inaccurate Accurate Accurate

161

104. Take no time for others. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 105. Break my promises. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 106. Am not bothered by difficult social Very Moderately Neither Accurate Moderately Very situations. Inaccurate Inaccurate nor Inaccurate Accurate Accurate 107. Like to take it easy. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 108. Am attached to conventional ways. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 109. Get back at others. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 110. Put little time and effort into my Very Moderately Neither Accurate Moderately Very work. Inaccurate Inaccurate nor Inaccurate Accurate Accurate 111. Am able to control my cravings. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 112. Act wild and crazy. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 113. Am not interested in theoretical Very Moderately Neither Accurate Moderately Very discussions. Inaccurate Inaccurate nor Inaccurate Accurate Accurate 114. Boast about my virtues. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 115. Have difficulty starting tasks. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 116. Remain calm under pressure. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 117. Look at the bright side of life. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 118. Believe that we should be tough on Very Moderately Neither Accurate Moderately Very crime. Inaccurate Inaccurate nor Inaccurate Accurate Accurate 119. Try not to think about the needy. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate 120. Act without thinking. Very Moderately Neither Accurate Moderately Very Inaccurate Inaccurate nor Inaccurate Accurate Accurate

162

APPENDIX D: PSYCHOMETRIC SCALE ITEMS

Dimension Factor Description Items Reverse Coding Related Literature All items measured on a Likert-type scale of 1 (P = positively through 5. 1 = Very Inaccurate, 2 = Moderately related to creative Inaccurate, 3 = Neither accurate nor inaccurate, 4 concept selection, = Moderately Accurate, 5 = Very Accurate R = negatively related) Personal Bias against The extent to which an individual I prefer creative designs over conventional P Rietzschel, E., et al. (2010). biases and creativity has an inherrent bias against designs "The selection of creative cognitive creativity. Research has shown that I believe that creative designs will lead to positive P ideas after individual idea style (34 people have this inherrent bias design outcomes generation: choosing between items) against creativity because of the I prefer conventional designs over creative R creativity and impact." British uncertainty regarding a novel designs Journal of Psychology 101(1): concept. I am skeptical that creative designs will lead to R 47-68. positive design outcomes Negativity Bias The extent to which flaws in I focus on the strengths of a design concept more P Amabile, T., & Glazebrook, A. potential design concepts are than the flaws of the design when making H. (1982). A negativity bias in emphasized and given more weight decisions interpersonal evaluation. than design strengths. Negative I believe that someone who gives negative P Journal of Experimental Social feedback or criticism is considered feedback is more intelligent and knowledgeable Psychology, 18(1), 1-22. more valuable and is than someone who gives positive feedback. Amabile, T. (1983). Brilliant overemphasized in the decision I pay more attention to design flaws than design R but cruel: perceptions of makers' mind. strengths in my decision-making. negative evaluators. Journal of I believe that someone who gives positive R Experimental Psychology, feedback is more intelligent and knowledgeable 19(2), 146-156. than someone who gives negative feedback Optimism An individual's belief that risky I believe that risky design concepts will lead to P Lovallo, D. P., & Sibony, O. decisions will produce positive positive design outcomes (2010). The case for outcomes I believe that risky design concepts will lead to R behavioral stategy. McKinsey negative design outcomes Quarterly, 2(1), 30-43. Hindsight bias The extent to which an individual I do not let my experiences with previous P Christensen-Szalanski, J. J., & believes that past experiences projects alter my perceptions and behaviors in Willham, C. F. (1991). The predict or heavily influence current future projects. hindsight bias: A meta-

163

or future events. The 'I knew it' I use my experiences with previous projects to R analysis. Organizational bias. alter my perceptions and behaviors in future Behavior and Human Decision projects. Processes, 48(1), 147-168. Receptivity of new The varying degrees of I am receptive to new ideas P Kaplan, Norman. "Some ideas receptiveness is based on the I will seriously consider novel ideas P organizational factors affecting individual's prior experiences and I am closed off to new ideas R creativity." Engineering inclinations. I disregard novel ideas R Management, IRE Transactions on 1 (1960): 24- 30. Ambiguity The extent to which an individual I am tolerant of ambiguous or unknown P Heath, C., & Tversky, A. Aversion is not tolerant of uncertain information during the design process (1991). Preferences and situations and tries to reduce the I embrace ambiguous or unknown elements in the P Beliefs: Ambiguity and uncertainty inherent in many real- early phases of design Competence in Choice Under life situations. I have no need to maintain a sense of certainty P Uncertainty. Journal of Risk during the design process and Uncertainty, 2, 5-35. I do not like dealing with ambiguous or unknown R Sorrentino R, Roney CJR elements in the design process (2000) The Uncertain Mind: I seek to reduce ambiguous or unknown elements R Individual Differenes in in the early phases design Facing the Unknown, vol 1. I feel the need to maintain a sense of certainty R Psychology Press, Hove, UK during the design process Cognitive way of Thinking is either convergent I try to find multiple solutions to a design P Guilford, Joy P. "Creative finding answers. (obtaining 1 right answer) or problem. abilities in the arts." divergent (obtaining a relative I believe that there are often multiple correct P Psychological review 64.2 answer). Creativity is influenced solutions to a design problem (1957): 110. more by divergent thinking. I try to find a single solution to solve a design R problem. I believe that there is usually one correct answer R to a design problem Risk preference An individual's preference for risk- I prefer taking risks during design projects P Sitkin, S. B., & Pablo, A. L. taking I avoid taking risks during design projects R (1992). Reconceptualizing the determinants of risk behavior. Academy of Management Review, 17(1), 9-38. Risk inertia An individual's tendency to take In the past, I have tried to take risks during P Sitkin, S. B., & Pablo, A. L. risks based on their prior risk projects (1992). Reconceptualizing the behavior. Researchers argue that if In the past, I have tended to stay away from R determinants of risk behavior. an individual has typically taken taking risks during projects Academy of Management risks in the past, they will be likely Review, 17(1), 9-38. to take risks in the future, and vice versa.

164

Outcome history An individual's personal experience I tend to have positive experiences when taking P Thaler, R. H., & Johnson, E. J. with taking risks. If an individual risks during projects (1990). Gambling with the experiences positive outcomes I tend to have negative experiences when taking R house money and trying to when taking risks, they are more risks during projects break even: the effects of prior likely to take risks again, and vice outcomes on risky choice. versa. Management science, 36(6), 643-660. Burden of proof The extent to which an individual I am easily convinced that risky design concepts P Mounarath, R., Lovallo, D., & requires 'proof' that a risky decision will be successful Dong, A. (2011). Choosing will pay off before making said I require proof or substantial evidence that a risky R innovation: How reasoning risky decision design concept will be successful before taking affects decision makers. Paper risks. presented at the International Conference on Engineering Design, Copenhagen, Denmark. Creative Self-efficacy for An individual's belief in their I am confident in my ability to identify when P Coopersmith S (1967) The confidence identifying and capability to identify and select ideas are creative Antecedents of Self-Esteem. and selecting creative creative ideas I am not confident in my ability to identify when R Freeman, New York. NY motivation ideas ideas are creative Fromm E (1959) The creative (25 items) attitude. In: Anderson HH (ed) Creativity and its Cultivation. Harper & Row, New York, NY, 44-54 Creative The extent to which an individual I believe that I am a creative individual P Phelan, S., & Young, A. M. Confidence/ believes that they are capable of I tend to pay more attention to creative ideas P (2003). Understanding purpose and intends to bring a novel, I tend to favor creative ideas P creativity in the workplace: An original, and creative idea into I intend to increase the creativity of the design P examination of individual being. process styles and training in relation I do not believe that I am a creative individual R to creative confidence and I tend to disregard creative ideas during design R creative self-leadership. I tend to not think highly of creative ideas R Journal of Creative Behavior, I am not focused on increasing creativity during R 37(4), 266-281. the design process Genuine sense of A person who is comfortable with I am comfortable with failures during the design P Rhodes, Mel. "An analysis of self and themselves and willing to accept process creativity." Phi Delta Kappan confidence. failure is a creative person. They I often feel confused or lost while tackling a P (1961): 305-310. have the ability to be puzzled and design problem the ability to accept conflict. I am comfortable with conflicting design P requirements I believe that I generate valuable ideas P

165

I am not comfortable with failure during the R design process I am seldom confused or lost while tackling a R design problem I am not comfortable with conflicting design R requirements I believe that I do not generate valuable ideas R A person's intrinsic Generating creative ideas requires a I am motivated to solve design problems P Andrews, Jonlee, and Daniel motivation to lot of time, effort, and ability to C. Smith. "In search of the perform the task. stay focused on the task at hand. A It is easy for me to stay focused on the task at P marketing imagination: person's investiment in the project hand during a design project Factors affecting the creativity will influence the quality of of marketing programs for I feel personally invested in the success or failure P solutions provided. mature products." Journal of of the outcomes in a design project Marketing Research (1996): I am not motivated to solve design problems R 174-187. It is hard for me to stay focused on the task at R hand during design projects I am not personally invested in the success or R failure of the outcomes in a design project Social Learning culture "The results indicated that both I feel that my current working environment P Ismail, Meriam. "Creative effects and and creative learning culture and creative encourages and fosters creativity climate and learning environmen climate. climate contributed 58.5 percent to I feel that my current working environment does R organization factors: their t (29 items) the explanation of the observed not encourage or foster creativity contribution towards variances in the innovation innovation." Leadership & construct."-Findings Organization Development Journal 26.8 (2005): 639-654. Sensitivity to The extent to which an individual I am usually unaware of when I am being P Sternberg RJ, O'Hara LA, crtiticism is dismayed and easily discouraged criticized Lubart TI (1997) Creativity as by criticisms to their ideas I am not easily discouraged when I am being P an Investment, California criticized Management Review, 40:8-21 I am usually aware of when I am being criticized R I am easily discouraged when I am being R criticized Freedom to The extent to which the individual I feel comfortable presenting my ideas to my team P Hoffman, L. Richard, Ernest express opinions feels comfortable expressing their members Harburg, and Norman RF ideas and opinions in a group I do not feel judged by my team members for my P Maier. "Differences and ideas and opinions disagreement as factors in I do not feel comfortable presenting my ideas to R creative group problem my team members solving." The Journal of I feel judged by my team members for my ideas R Abnormal and Social and opinions Psychology 64.3 (1962): 206. 166

Competition Wanting to outperform others can I have a competitive nature P Paulus, P. (2000). Groups, lead to one expressing ideas more I want to outperform others P Teams, and Creativity: The freely and therefore expressing I am not competitive R Creative Potential of Idea- more creative ideas. More likely to I do not feel the need to outperform others R generating Groups. Applied take a risk if it will give them the psychology, 49(2), 237-262. upper hand in a project fear of rejection people with certain personalities I like to take ownership of my ideas and opinions P Paulus, P. (2000). Groups, and failure are more likely to be more creative I tend to express my ideas freely P Teams, and Creativity: The when they aren’t afraid of their I tend to handle failure well P Creative Potential of Idea- creative ideas being rejected I prefer to submit my ideas anonymously R generating Groups. Applied I am not comfortable expressing my ideas freely R psychology, 49(2), 237-262. I do not cope well with failure R Opinion of how much does someone else's I prefer to go against the grain P Sternberg, R. J., O'Hara, L. A., others/assessment opinion of you influence your I often have unique ideas and opinions P & Lubart, T. I. (1997). by peers decisions I do not follow trends P Creativity as investment. California Management I am not intimidated by other people's opinions of P Review, 40, 8-21. me People's opinions of me seldom affect my own P actions I prefer to not go against the grain R I often have conventional ideas and opinions R I often follow trends R I am often intimidated by other people's opinions R of me People's opinions of me often affect my own R actions Assessment by How focused one is on their grades I want to impress my superiors (supervisor, P Davies, T. (2000). Confidence! superiors or what their boss will think of instructor, professor, leaders) Its role in the creative teaching them I want to get good grades P and learning of design and It is not important for me to impress my superiors R technology. (supervisor, instructor, professor, leaders) I do not care about getting good grades R Network/ Team The distance between the I play a central role in teams that I am a part of P Perry-Smith JE (2006) Social Centrality individual and the rest of the design I believe that I influence the direction and P yet creative: the role of social team. High centrality indicates that progress of projects that I am a part of relationships in facilitating the number of links required to I do not typically play a central role in teams that R individual creativity, Academy access other members of the I am a part of of Management Journal, network are few, whereas low I believe that I have little impact on the direction R 49:85-101 centrality indicates that the number and progress of projects that I am a part of of links required to access other members are many. 167

Personality Extraversion Energy, positive I am the life of the party P McCrae R (1987) Creativity, traits emotions, urgency, assertiveness, I talk to a lot of different people at parties P divergent thinking, and (FFM) 16 sociability and the tendency to I keep in the background R openness to experience, items seek stimulation in the company of I don't talk a lot R Journal of Personality and others, and talkativeness. Social Psychology, 52:1258- Agreeableness A tendency to I sympathize with others' feelings P 1275 be compassionate and cooperative r I feel others' emotions P ather I am not really interested in others R than suspicious and antagonistic to I am not interested in other people's problems R wards others. It is also a measure of one's trusting and helpful nature, and whether a person is generally well tempered or not. Conscientiousness A tendency to be organized and I get chores done right away P dependable, show self-discipline, I like order P act dutifully, aim for achievement, I often forget to put things back in their proper R and prefer planned rather than place spontaneous behavior. I make a mess of things R Neuroticism The tendency to experience I have frequent mood swings P unpleasant emotions easily, such I get upset easily P as anger, anxiety, depression, I am relaxed most of the time R and vulnerability. Neuroticism also I seldom feel blue R refers to the degree of emotional stability and impulse control and is sometimes referred to by its low pole, "emotional stability". Openness/ Intellect Appreciation for art, emotion, I have a vivid imagination P adventure, unusual ideas, curiosity, I am not interested in abstract ideas R and variety of experience. I have difficulty understanding abstract ideas R Openness reflects the degree of intellectual curiosity, creativity and a preference for novelty and variety a person has. It is also described as the extent to which a person is imaginative or independent, and depicts a personal preference for a variety of activities over a strict routine.

168

[email protected] personal.psu.edu/cot5015 Christine (814) 769 0379 Toh Industrial and Manufacturing Engineering The Pennsylvania State University 343 Leonhard Building, University Park, PA 16802

EDUCATION Ph.D., Industrial Engineering | Anticipated Graduation: August 2016 The Pennsylvania State University, University Park Dissertation: Towards a Theoretical Understanding of Creative Concept Selection in Engineering Design M.S., Industrial Engineering | 2014 The Pennsylvania State University, University Park Thesis: Understanding The Factors That Influence Creative Idea Development. M.S., Mechanical Engineering | 2014 The Pennsylvania State University, University Park Thesis: The Impact Of Product Interactions, Creativity In Engineering Design B.S., Mechanical Engineering with a minor in Psychology | 2011 The Pennsylvania State University, University Park

RELEVANT Graduate Research Assistant, Ph.D. in Industrial Engineering | Fall 2011 – Present EXPERIENCE Advisor: Dr. Scarlett Miller Sponsor: The National Science Foundation • Investigated the role of creativity and decision-making biases in concept selection during engineering design using quantitative and qualitative studies with engineering students • Developed and conducted online studies with engineering design professionals to explore perceptions of creativity and the use of formalized concept selection tools in design industry • Explored the role of virtual learning platforms in the engineering classrooms through qualitative studies with engineering students Human Factors Engineering Intern | Summer 2012 Videon Central Inc. • Analyzed the usability of a multimedia application for cross-platform use and developed recommendations for improving the interface, a majority of which have been implemented in later releases • Conducted market analysis and customer needs assessment to improve the focus of the application • Collaborated on back-end computer programming tasks on Android platforms

PROFESSIONAL Reviewer, ASME International Design Engineering Technical Conferences | 2013, 2014, 2015 SERVICE AND Academic Integrity Committee Member | Fall 2015- Present ACTIVITIES College of Engineering, Penn State University LGBT Student Mentor | Fall 2011 - Present LGBT Student Resource Center, Penn State University Graduate Student Mentor | August 2014 ASME International Design Engineering Technical Conference, Buffalo, NY Member, American Society of Mechanical Engineers | Fall 2010 – Present Member, Human Factors and Ergonomics Society | Fall 2013- Present Member, Out in Science, Technology, Engineering, and Mathematics | Fall 2009 – Present

ACADEMIC Best Presentation Award | 2015 HONORS AND College of Engineering Research Symposium, Penn State University AWARDS Best Poster Award | 2014 College of Engineering Research Symposium, Penn State University Lagoa, Ray and Monkowski Graduate Award for excellence in scholarship and research | 2013 Penn State University Best Paper Award | 2013 College of Engineering Research Symposium, Penn State University

TECHNICAL Languages | Visual Basic, C, R, MATLAB, LaTEX PROFICIENCIES Application Software | SPSS, Minitab, Balsamiq, Adobe Creative Suites, Arduino Programming