Divergent Thinking in a Neurodynamical Model of Ideation Mei Mei and Ali A. Minai, Senior Member, IEEE Abstract—Divergent thinking refers to a style of thinking that be able to make more unusual conceptual combinations than ranges across a broad range of concepts, and is considered to the former. be a core enabler of creativity. Thinking is often modeled as a Conceptual associations – or, more accurately, word asso- process of conceptual combination, and creative ideas are seen as those using unconventional combinations of concepts. Since ciations – have been studied extensively through the exper- conceptual combination is fundamentally an associative process, imental determination of association norms [7] or analysis it has been proposed that creative thinking reflects stronger than of associations using dictionaries and thesauri [8], which expected associations between normally remote associates in the provide valuable information on objective patterns of associ- mind of the thinker. We recently proposed a neurodynamical ation in a reference populatio, e.g., modern English speakers. model called Itinerant Dynamics with Emergent Attractors (IDEA) to explore the process by which ideas can emerge in Associative recall in individual subjects has been studied an associative memory system through conceptual combination. through priming and cued recall experiments [9], [10], [11], In this paper we study how the functional dynamics of this [7], and recently through experiments designed to explicitly model changes when its associative weights are changed to make map the patterns of association in individuals [12], [13]. remote associations stronger. We apply the model to data from Mednick’s hypothesis has also been evaluated experimentally four sets of papers from previous IJCNN meetings. In addition to identifying the differences in the dynamics due to changes in with mixed results [14]. However, none of these approaches the association pattern, we consider whether divergent thinking completely captures the functional strength of associations can potentially predict future ideas or reconstruct past ideas. in thinking, which is fundamentally a dynamical and fluid process with one thought generating the next. Concepts occur I. INTRODUCTION in this stream based not on their pairwise associations with other concepts but on their joint participation in complex The process by which ideas – especially novel ideas – ideas involving multiple concepts. We recently introduced a are generated within the mind has long been a topic of neurodynamical model called itinerant dynamics with emer- interest to philosophers and scientists. Within the framework gent attractors (IDEA) to model this process [15], [16], [17], of modern psychology and cognitive science, a widely held [18]. In this model, a recurrent semantic neural network assumption is that ideas emerge through the combination is built from empirically determined associations between of existing concepts, and that novel ideas represent new, concepts, and the dynamics of the network then generates an unexpected combinations [1], [2], [3], [4], [5], [6]. The key itinerant sequence [19] of metastable attractors representing factor underlying this process of conceptual combination is combinations of mutually associated concepts. While the association. The mind’s conceptual space can be seen as associations used to instantiate the model could come from an associative memory encoding complex patterns of useful any valid source, in most of our work they have been derived associations between concepts from multiple modalities and from corpora of continuous text or speech produced in at multiple levels of representation. The process of generating natural settings. The assumption is that the correlations found thoughts or ideas can then be considered a dynamical recall through such data represent actual associations underlying process that unmasks conceptual groups with sufficiently fluent thought, and are thus a more accurate reflection of as- strong mutual associations. In a seminal paper, Mednick [2] sociations in the minds of the authors or speakers. The IDEA proposed that an individual’s capacity for creative thinking model has been applied to identifying significant conceptual depended strongly on the structure of their conceptual asso- combinations in texts such as conference proceedings [20], ciations. He argued that, for any given concept, most people and to identifying semantically salient words in several types have only a few strongly associated concepts, but creative of text [21]. individuals have relatively strong associations with a larger In our previous work, the weights of the network were set of concepts that would generally be considered unusual. based purely on measures of association such as co- For example, most people may asscociate the concept “table” occurrence, correlation or pointwise mutual information be- with “chair”, “top”, “dining” and “coffee”, but someone with tween words in the reference corpus [22]. In this paper, we more grounding in science and mathematics would also think consider how a nonlinear transformation of these weights of the periodic table from chemistry, the water table from to model divergent thinking influences the dynamics of the geology, truth tables from logic, etc. The latter person would system, and whether it allows the generation of more wide- ranging – possibly even predictive – ideas. Mei Mei (Email: [email protected]) and Ali Minai (Email: [email protected]) are all with the Department of Electrical Engineering II. BACKGROUND AND MOTIVATION and Computing Systems, University of Cincinnati, Cincinnati OH 45221 Acknowledgement: This work was supported in part by a National Science The cognitive mechanisms and neural processes underly- Foundation INSPIRE grant to Ali Minai (BCS-1247971). ing creative thinking have been an object of fascination and study for a long time. The modern tradition of such studies corpus of a poet or single books on a specific topic as sources goes back of Guilford [23] and Hebb [24], who proposed of data [18], [22]. An especially interesting and useful that new ideas emerged through the combinatorial activity of source of data is corpora of papers published in technical neuronal assemblies. Since then, there have been numerous conferences or journals. While these typically involve many behavioral studies [25], [26], [27], [28], [29], [30], [31], authors, they can be seen as representing the “collective [32] as well as neurobiological studies using brain imaging mind” of an epistemic community, and application of a methods [33], [34], [35], [36], [37], [38], [39]. They indi- model such as IDEA to this data can be used to identify the cate that creative thinking arises through a combination of core ideas within this community. We have previously used semantic associations and cognitive control [37], [40], [14], abstracts from a set of IJCNN meetings for this purpose [20], [38]. Many of the influential theoretical models of creative [21]. In this paper, we extend this to much larger corpora thinking have postulated that associative recombination of built from full papers across a number of IJCNN meetings. concepts is the main mechanism for generating novel ideas This paper focuses on two main issues: 1) How does [1], [2], [3], [5], [4], [32], [6], though processes such as modifying the connectivity of the epistemic network in the generalization, off-label use and analogy have also been IDEA model in ways consistent with Mednick’s proposal implicated [41], [42], [43]. Various computational models [2] change the dynamics of conceptual combination? 2) Can have also been proposed to study, understand and possibly the IDEA model – especially in divergent mode – predict enhance creative thinking [44], [3], [41], [45], [42], [43], future ideas in an epistemic community based on associations [46], [47]. derived from past ideas. Clearly, the latter issue is highly A key element in many theoretical models of creativity speculative, but any progress on it can potentially be of great is the idea that the pattern of conceptual associations in value. the thinker’s mind is an important determinant of creativity [2], [42], [12], [13]. The use of associative neural networks III. DATASETS to model creativity has been a central feature of models We used proceedings from several years of the Interna- developed by our research group [48], [49], [15], [46], [50], tional Joint Conference on Neural Networks (IJCNN) to [51], and others [47]. The IDEA model [16], [17], [18], [20], create the datasets. The datasets were created by taking only [21], [22] forms the core element of our overall model for the main body of papers from the proceedings, removing creative thinking, and is based on the following postulates: figures, equations, citations, etc., and representing each paper 1) Semantic knowledge is represented in the brain through as a sequence of sentences. Stop words were removed using associations between conceptual elements represented a standard list, and the other words were stemmed to group as neural units, forming an epistemic network. together variants of the same root, e.g., “test”, “tested”, 2) Ideas arise in the epistemic network as emergent attrac- “testing”, etc. The data were then filtered further to remove tors – metastable activity patterns – through itinerant non-salient words using methods described in our previous neural activity dynamics [19]. The activity of the work [20]. network lingers around an emergent attractor for some