Conceptual Combination versus Critical Combination: Devising Creative Solutions using the Sequential Application of Crowds Yuko Tanaka ([email protected]) Howe School of Technology Management, Stevens Institute of Technology Castle Point on Hudson, Hoboken, NJ 07030 USA Yasuaki Sakamoto ([email protected]) Howe School of Technology Management, Stevens Institute of Technology Castle Point on Hudson, Hoboken, NJ 07030 USA Takashi Kusumi ([email protected]) Graduate School of Education, Kyoto University, Yoshida Hon-machi, Sakyo-ku, Kyoto 606-8511 JAPAN Abstract goods and services has been regarded as a viable alternative to firms and markets (Benkler, 2006; Howe, 2006). The purpose of this study was to examine the steps involved in the sequential application of crowds to produce creative In our past work (Nickerson & Sakamoto, 2010), we have solutions for social problems. 127 university students examined one way of using crowds to design creative ideas. generated, criticized, modified, and evaluated text solutions The basic procedure was to ask the first crowd to generate a for reducing misinformation on the Internet. One crowd set of ideas, akin to divergent thinking, in which diverse generated solutions based on two ideas randomly paired from ideas are generated. The second crowd combined two ideas previous crowds’ ideas. The other crowd came up with from the first crowd, akin to convergent thinking, in which solutions based on one idea and a criticism for this idea generated by previous crowds. Whereas solutions based on ideas are filtered and integrated. Then the third crowd two ideas were on average rated more creative than those combined two ideas from the second crowd, in the hopes of based on an idea and criticisms, the five most creative further improving the creativity of the ideas. solutions that addressed criticisms were rated significantly Letting crowds combine ideas is a reasonable approach more creative than the five most creative ideas that combined for improving the creativity of the variants. Cognitive two ideas. We conclude that a critical crowd can be useful if scientists have proposed that creativity depends on the goal is to collect a small number of highly creative ideas conceptual combination, in which separate ideas or concepts using the sequential application of crowds. are merged (e.g., Ward, 2004). According to Rothenberg Keywords: Creativity; wisdom of crowds; collective (1979), creative accomplishments, such as original arts and intelligence; critical thinking scientific discoveries are the results of synthesizing two opposing ideas. Thagard and Stewart (2011) propose that Introduction novel combinations of mental representations underlie many Advances in communications technologies have changed kinds of creativity. the way people collaborate. In many websites, crowds work In crowd creativity, the results from the past work are in isolation for the most part, and their ideas are aggregated mixed. For some tasks, such as designing children’s chairs later. For example, users may rate news stories posted on a in drawings, letting crowds combine previous designs website, and the website may show the aggregate rating for resulted in more creative designs (Yu & Nickerson, 2011). each story. There are many web services for collaborative For other tasks, such as generating solutions for oil-spill tagging and filtering, online auctions, and prediction problems in texts, there was little if any improvement in markets, which allow users from afar to collaborate creativity of ideas generated by later crowds (Bao & efficiently without knowing the others. Sakamoto, in preparation). Perhaps combining two text The current work examines how crowds can be used to ideas for solving social problems is difficult, especially collect creative ideas. In particular, we examine the steps when the two ideas are non-alignable and there is no easy involved in the sequential application of crowds to design way to integrate elements from different ideas (cf. Markman solutions for social problems. We think the use of crowds et al., 2009). Such situations are common with text ideas for has a great potential for solving many difficult problems. solving social problems such as dealing with oil spills, but Although online crowds work for free or for nominal fees, less common with designs of chairs. collective solutions of crowds can be as useful as those of a An alternative to combining ideas is to let the crowds few experts (Surowiecki, 2004). Aggregation cancels out address criticisms of ideas. A critical crowd can find a individual errors and highlights “correct” solutions (cf. problem of an idea, and the next crowd can address the Estes & Maddox, 2005; Galton, 1907), leading to the problem. Defining a problem, which is an important wisdom of crowds. Moreover, the idea of crowds generating component of critical thinking (Ennis, 1987), interacts with 3116 generating ideas in creative problem-solving (Treffinger, et. 2. Crowd 4 evaluated 180 ideas by Crowd 3 on novelty and al., 1994). In this way, critical thinking is related to practicality, which are two important components of creativity (Brookfield, 1987; Moon, 2008). By addressing creativity (Runco & Pritzker, 1999), and are often used in criticisms, the crowd will refine and elaborate on an idea, measuring creativity (cf. Dean, 2006; Finke et al., 1992). In which will improve the quality of the idea (Runco & crowd 2 and crowd 3, participants were randomly assigned Pritzker, 1999). Thus, we hypothesize that including a to either the idea-idea or idea-criticism condition. crowd, which concentrates on finding problems into crowds will contribute to generating creative ideas. Table 1: The design of the current study Like the past work, the first crowd generates a set of ideas. Instead of combining two ideas from the first crowd, Crowd Idea-idea Idea-criticism the second crowd generates criticisms for given ideas. The 1 Generated 30 ideas third crowd generates a new set of ideas based on an idea (n=10) from the first crowd and its criticism from the second Generated 30 criticisms 2 crowd. By addressing the criticism of the idea, the third Generated 30 ideas of 30 corresponding (n=10, 10) crowd will refine and elaborate on the given idea. Here we ideas by Crowd 1 compare this sequence involving a critical crowd, which we call the idea-criticism condition, with the sequence from the Generated 90 ideas Generated 90 ideas based on 30 pairs based on 30 pairs of an past work involving only combination crowds, which we 3 of an idea by idea by Crowd 1 and call the idea-idea condition. (n=30, 31) Crowd 1 and an its criticism by Crowd One theoretical contribution of our work is that we apply work in critical thinking and conceptual combination to idea by Crowd 2 2 crowd. In practical terms, by better understanding the 4 Evaluated 180 ideas by Crowd 3 processes of crowd creativity, we can help design social- (n=36) on novelty and practicality computational systems that are tailored to the goals of the systems. In terms of societal impacts, if the use of crowds in Crowd 1 Each participant was asked to generate three ideas. generating creative solutions works, we may be able to find For each idea, the participant filled in the three blanks: “One creative solutions for many challenging social problems. way to avoid or reduce the negative influences of misinformation on Internet users is to [blank]. An example Method is to [blank]. The advantage of this is that [blank].” Participants typed their answers into an Excel spreadsheet. Participants In total, Crowd 1 generated 30 ideas. In total, 127 Japanese undergraduate and graduate students Crowd 2 In the idea-idea condition, each of 10 participants (60 men, 67 women) at Kyoto University participated in was asked to generate three ideas in the same fashion as four crowds, with the mean age of 22 (SD = 3). Each those in Crowd 1, resulting in 30 new ideas. participant was in only one crowd. Each participant received In the idea-criticism condition, each of the other 10 a bookstore gift card in the amount of 500 Japanese yen participant was shown three ideas generated by Crowd 1, (about $6) after completing the task. one at a time, and asked to point out a problem of each idea by filling in the blank: “The problem of this idea is that Apparatus and Materials [blank]. Therefore, it is difficult to avoid or reduce the negative influences of misinformation on Internet users.” Data were collected using Excel spreadsheets on a 15-inch This resulted in 30 problems associated with 30 ideas by monitor in a laboratory. The following social problem was Crowd 1, one problem for each idea. Participants were also used: “There is diverse information on the Internet, asked to explain the reasons behind their responses. including wrong information, deceptive information, and Crowd 3 In the idea-idea condition, each idea by Crowd 1 information that is not based on fact. What can we do to was paired with a randomly selected idea from Crowd 2 of avoid or reduce the negative influences of misinformation the idea-idea condition. Each of 30 participants was asked to on Internet users?” generate a new idea based on one of the 30 pairs. This Design and Procedure procedure was repeated three times for each participant. In the idea-criticism condition, each idea by Crowd 1 and Table 1 summarizes the design of the current study. There its criticism by Crowd 2 of the idea-criticism condition were were four crowds. Crowd 1 generated 30 ideas. In the idea- presented together. Each of 31 participants was asked to idea condition, Crowd 2 generated another set of 30 ideas, generate a new idea based on one of the 30 pairs. This and Crowd 3 generated 90 ideas based on 30 pairs of an idea procedure was repeated three times for each participant.