arXiv:2008.05959v1 [cs.CY] 13 Aug 2020 uin[6.Tegoigitrs o hs susi re- on is efforts issues research these models widespread for the interest in growing flected evo- The intellectual and its social [26]. hand, our lution other to phe- central the cognitive are On model that to nomena requires [49]. study life of object of principal tree the other of from beings branches human distinguishes fundamentally so rca hog w anapcs nteoehn,it hand, one the On of understanding aspects. the be involves main to proves two behavior through human intricate crucial more this of study h iisaddfiiniso urn I eurn omove to requiring towards AI, current of deficiencies underline and we limits concept, the context-prone As highly lens. a this is through creativity considered be would on AI vie rely current to creativity how we between of so, studies frontier science do social the To from reflections blurring sciences. by cre- computational AI, of and of question social era the the on perspective at new ativity a we provide paper, this to In aim endeavors. creative for implication its debates on as unhinged somewhat labeled and artefact However, passionate unleashed technological has century. a this of of aura trend the natural artistic a the in appear to creativity continuity for technologies such The of applications. use various across successes proficient seen ormi a rmunderstanding from far remain to of field the around learning crystallized deep recently field, thrilling this constant in the with leaps Even goals. well-defined of set a on focused logical has (AI) Intelligence resear scientific Artificial to contributions substantial its Despite agents. sheer artificial the of by power offered computational possibilities unlimited almost harness the not ing of risk at us putting counterproductive, highly ftyn oprl ii ua raietat towards traits creative human self-contained mimic a purely to trying of In- Artificial of through advent (AI), the telligence for Recently, tools novel of purposes. plurality evolution creative a the of provided these, have variety Along technology extensive of era. an that with to relevant infused meanings been has different it Across epochs, humanity. our to quintessential arguably raiiyi epydbtdtpc sti ocp is concept this as topic, debated deeply a is Creativity prahs iigt ov omlpolm with problems formal solve to aiming approaches, o ievreyo ak rmdvrehorizons, diverse from tasks of variety wide a for rica creativity artificial RAIIYI H R FATFCA INTELLIGENCE ARTIFICIAL OF ERA THE IN CREATIVITY h urn tt forkoldeseems knowledge our of state current the , .INTRODUCTION 1. ex-nihilo A BSTRACT eplearning deep RA NSUR91 TS obneUniversite Sorbonne STMS, 9912 UMR CNRS - IRCAM eeaiemciewudbe would machine generative eageta h objective the that argue We . creativity raieendeavors creative prahs have approaches, hsapc that aspect this , hlpeEln,NnnDevis Ninon Esling, Philippe mathematico- [email protected] generative intelligent The . ch, w - oe fmcie.Suyn hs usin ol fos- could questions these Studying machines. of computationa sheer power almost the the by harnessing offered not possibilities of unlimited risk counter- at be us would putting behavior productive, cognitive on our human based purely to of approach mimicry complementary an Hence, are limits. and and flaws limits, own propen- our a expand have computers to that sity argue we AI, future inher- for in the contrived researches evaluate be could to path try what we and As limitations, intel- ent AI. of an era as the of in seen creativity understanding be current our can reflecting approach experiment This lectual creativity. stud- of psychological ies and a cognitive across of sci- AI review of computational concomitant properties and mathematical infusing social to by between seek ences we frontier Hence, the creativity. blur through reification, of AI aspects of of sociological that aspects the mathematical to scrutinize stance to different aiming a taking thought, knowledge. our cognitive by limited stranded ap- and is observations reification it current this as However, limited, inherently acces- appears thoughts. are proach own that our objects to conceptual on sible reason perceiv’d only not can has we he what desire can As none to reach. can bound perception " are wrote own Blake humans, our that as ideas We, only address field. appears this which to systems, foundational biological in witnessed behavior path, own its hypotheses carved the have term to we seems which thought of line other recycling ative aebe hijacked been aspects have com- problem-solving Historically, for developed view. models major be putational two can into creativity divided computational broadly per- in epistemological approaches an From spective, hu- of interactions. metaphor creative powerful man a phys- shaping and profoundly sensitive one, hu- abstract, ical complex an being and while of interactive activity, one man organized, is music highly as most general, the in remark- creativity a on provide projector also able can mechanisms creative thes musical revealing Hence, goals. task-oriented through hardly defined objectives, unsupervised model. on to operates strenuous music are Indeed, that processes que cognitive theoretical be- and stimulating tions creative together brings of it comprehension as haviors, our developing for work [35]. sciences fundamental to industrial spanning nti ae,w r osrl onanvlpt of path novel a down stroll to try we paper, this In mdtteeqetos ui rvdsa da frame- ideal an provides music questions, these Amidst ned Ii tenb iir fintelligent of mimicry by strewn is AI Indeed, . a’ eie r iie yhsperceptions; his by limited are desires Man’s oee,wti h Irvlto,an- revolution, AI the within However, . posteriori a ahmtclricto fcognitive of reification mathematical yatssa omof form a as artists by 1] Hence, [11]. " cre- s- l e ter and enrich the relationship between humans and AI will gradually be integrated as an impending norm of the by targeting situations of partnership converging towards era, slowly becoming mundane in the society and domain more symbiotic co-creative interactions. Hence, address- it operates [18]. An emblematic example in the musical ing these questions could give rise to a novel generic cat- realm is that of musical synthesizers [46]. When they egory of creatively intelligent systems. appeared, audio synthesizers remained for long confined to the fringe of contemporary and experimental music, producing unheard sounds that were too unsettling to be 2. EPISTEMOLOGY OF CREATIVITY deemed musical yet. It is only through a gradual evolu- tion period that these sounds emerged across almost all The notion of creativity, core to this paper reflection, is musical genres, which now commonly integrate different a topic that could hardly be more central and inseparable levels of audio synthesis. In that sense, creativity is a from our humanity. In all human crafts, arts and science transformative process, adapting and integrating elements alike, progress seems to be rhythmed by the pace of our to a domain with predefined norms, steadily shifting and capacity to divorce ourselves with the present, to reinvent adapting these norms [45]. and overtake existing thought patterns, and create novel In summary, although creativity is complex and multi- ones. While appearing so quintessential and fundamen- faceted, it can be articulated around three major compo- tally distinguishing human beings from other branches of nents of novelty (creative ideas are innovative), quality the tree of life, creativity might be the most prominent ex- (appeal of the idea) and relevance (the idea is appropri- ample of a mental phenomenon that is so central to our ate to the task and era) [33]. Hence, studying creativity own existence, yet we understand so little about it [21]. requires to consider a large number of nuances, which are themselves mostly subjective and renders the analysis of 2.1. Defining creativity creativity as a set of empirically testable hypotheses rather tedious. This might explain why the empirical study of As we aim to reason on conceptual objects, understand- creativity is only at a very precocious stage. ing creativity requires first the ability to precisely define it. However, the concept of creativity appears to be partic- 2.2. Historical aspects ularly multi-faceted and complex to define. As is usually the case with such highly abstract concepts, decades of de- There is a strong historical component to the development bates have first focused on carving out its uniqueness by and social perception of creativity, for long stranded be- delineating it from closely-related concepts such as origi- tween notions of madness or genius. Different eras and nality, genius, imagination and talent [49]. In the collec- societies have been more or less conducive to flourish- tive subconscious, the core aspect of creativity lies in nov- ing creativity, with blossoming periods such as Ancient elty. In that sense, creativity involves moving across our Greece, Italian Renaissance or French Lumiôlres. To un- preconceived knowledge and creating a schism between derstand these disparities, we provide a scarce outline of the present and the future [16]. Through this first lens, the conceptual evolution of creativity along history, based creativity seems to put a strain on our relationship with the excellent work of Runco [49]. For a long time, West- the future, as it introduces uncertainty. Embracing cre- ern societies only considered creative traits in the artis- ativity implies a hazardous leap forward that might upset tic domain, with a widespread predilection for the myth the balance of our pattern-seeking habits, as it is impos- of the "lone genius". Overlaying the strongly theistic vi- sible to fully understand the consequences of an entirely sions of societies at that time, creativity was viewed as novel concept or object [41]. However, we might take a divine intervention, manifested as an outside "spirit" or comfort in the fact that this uncertainty is not boundless, "muse" for which the individual creator was merely a con- as creativity (and art alike) does not arise from a concep- duit [49]. This vision started to shatter, when the scien- tual void [37]. Even though creativity lives in the realms tific era blossomed around the 18th century. This most of the least predictable concepts, it remains first and fore- influential event in the history of creativity, consecrated most a contextually-embedded phenomenon [38] that all people exhibit different levels of talent in the wake Indeed, creativity involves social aspects, as it impli- of education, and that the "original genius", which was a cates individuals in a context, working with a set of ex- form of rule-defying exception [22], was divorced from isting patterns of meanings and symbols at a given time the supernatural. In response to the industrialization of in history [28]. The significance and relevance of creative Europe, Rousseau and the Romanticism movement ex- ideas is not solely observed in their content, but within pressed a separation between the scientific rationalism and the social framework and historical period at which these the need for humans to rely on their natural feelings as a ideas are produced. Ideas appear relevant only when a source of wisdom. This vision created a schism in the so- group of persons articulate their thoughts around the same cietal vision fracturing the rational scientist with the mis- set of questions [49], and a critical mass of knowledge de- understood deviant artist [49]. This might have created velops in one place. Hence, creativity offers variation and a paradoxical prejudice allowing to denigrate artists and depends equally on the properties of the environment as creativity as being confined to deviant personalities. This on its own quality [17]. Conversely, what was deemed rev- misconception unfortunatelysomehow survives in the col- olutionary and obtained widespread acclaim at one point lective unconscious to this day. Although creative persons and inventors are touted as a driving force to the improve- 2.3.1.3. Creativity as transformation of expertise ment of society [41], this image of deviance still allows The two roles of creativity in society can interact and for the denigration of artists when need be. produce, for instance, improvement as a result of self- expression. In any case, it is important to note that cre- 2.3. Roles and construction ativity is before all a transformational activity. There is a widely agreed-upon consensus that creativity is only per- A major question lies in the role that creativity plays in mitted through an existing body of knowledge, strongly a given society, and why creativity might be so impor- influencing the natue and quality of creative outcomes. tant. This outlines the perceived impact of creativity, but As stated previously, the uncertainty of creativity is not also what aspects are valued in creativity. As discussed boundless, as nothingarises from a conceptualvoid. When by Gardner [27], creativity can be seen as a temporary Newton stated "If I have seen further it is by standing on misalignment between an idea and the society in which it the shoulders of giants.", he recognized that his own dis- develops. Eventually, as some people are willing to take coveries were only made possible by all the knowledge more risk and embrace new ideas, these gradually inte- accumulated by previous researchers before him. Hence, grate the social fabric and ultimately become an accepted creativity is the result of community-built expertise, later and standardized part of society. transformed in an incremental fashion

2.3.1. Creativity in society 2.3.2. Evolution and construction Moran [41] proposed to study the functional aspects of the role that creativity could play in society, proposing that it This transformational view is epitomized in the evolution- mostly endorses functions of improvement or expression. ary theory of creativity, which proposes an interesting par- The improvement role is the effect that a creative object allel between genetic evolution and the development of can have on society (as technological artefacts), while the creative ideas. The original Darwinian model of Simon- expression role is focused on the role that creativity exert ton [53] aimed to describe more developmental aspect of on an individual. Although these two roles seem some- the creative process, but it naturally extends to larger (so- what dichotomous, they are proposed to interact in more cietal) scales of how creativity developed across time and of a complementary than competitive way. how social factors can come into play. Indeed, creative ideas are built on previous ones in an adaptive and open- 2.3.1.1. Creativity role as improvement ended manner [26]. This view of creative ideas evolving The vision of creativity as improvement is that which over time through culture can be depicted in Darwinian is often glorified by political or industrial leaders, as a terms [6], as pertaining to a form of inheritance of ideas, mandatory tool for the advance of society and humanity. which are incrementally adapted to the timely constraints In that sense, society is seen as a system, which is con- of their social environment as they pass from one person stantly moving upwards to an hypothetical blessed state to the next. This concept is termed as the dual-inheritance (notwithstandingour complete absence of a shred of knowl- theory [26], which emphasizesthe fact that we inherit both edge to where that might be). Hence, creativity is the tool biological but also cultural information. This theory views that transport society across the borders of the present to- culture as discrete elements, which are submitted to an wards an idealized future, by shifting norms to a higher adaptation process, both composed of random mutations position [49]. Whereas practitioners develop norms, cre- and a fortuitous process where ideas are selected because ativity pulls society forward, while inevitably giving way of outside environmental effects [20]. to standardization [41]. In that view of creativity as an In that sense, we explore an unknown space of ideas enhancer of society, its importance is the goal and that it (variation) and choose to pursue some and not others (se- allows us to progress towards it. lection), turning creativity in a variation-selection algo- rithm informed by expertise [21]. Here, the social con- 2.3.1.2. Creativity role as expression text is of prime interest, as creative ideas are observed The other role of creativity proposed by Moran [41] equally for their content and within the social framework is that of expression, which might be interpreted differ- within which these are produced. As ideas appear rele- ently depending on the society in which it unfolds. In vant only when a critical mass of knowledge is articulated that aspect, creativity can be regarded as a mean of self- around similar thoughts, this further underlines the ubiq- expression and individuality, while the exact nature of this uitous need for knowledge, which is transformed through expression dependson the permissiveness of the surround- creative processes. However, this thickens the complexity ing society. These forms of creativity in society are less of clearly separating intelligence and creativity. Indeed, focused on their results, and rather allow for solipsistic intelligence can arguably be described as the process of and individualistic approaches. Hence, creativity as ex- associating and transforming existing knowledge. Yet, as pression can be seen as a cathartic activity, which might social scientists debated for over a century to delineate be less valued by society. these two behaviors, this warrants the existence of a sim- ilar need for moving our reasoning from artificial intelli- gence towards artificial creativity. 2.4. Towards the existence of artificial creativity posed model. However, these limitations are partly by- passed in the AI domain by using approaches based on Being able to truly delineate intelligence and creativity probabilistic formulations [8], where we try to understand would mean that they conceptually live as independent or- the entire distribution of given types of data, rather than thogonal dimensions, which can each be evaluated sepa- answering a question (such as classifying objects). rately from the other. Hence, this poses the question of the measure and evaluation of creativity as a dimension If we take a closer look at the criteria that are used dissociated from intelligence. for divergent thinking tests, these are usually evaluated on several indicators measuring the ideational fluency (amount of answers), originality (unusualness of answers), flexibil- 2.4.1. Evaluating creativity ity (variance in the concepts elaborated) and elaboration The question of assessing creativity might be one of the (precision and details of the answers) [43]. A criticism of most controversial and complicated issue of the field. This this evaluation method is that fluency can act as a contam- approach (called psychometric) is unique as it also radi- inating factor in the originality scores, emphasizing the ates and impact all other types of creativity studies [36]. quantity over quality of ideas [43]. This entails the sensitive question of measurement, which By construction, probabilistic modeling could easily appears at first to be dauntingly complex in creative fields. overpower any human on these tests for fluency and flex- Yet, this field has a very extensive and flourishing research ibility. Indeed, an approach based on probability distri- history [44], entailing the reliability (agreement and con- bution estimation can produce an infinite number of solu- sistency), validity (accuracy of the measurements) [36], tions and these can be as distant as we want. However, and discriminant validity (not being contaminated by cor- there is two caveats to this reasoning. First, this implies related concepts of intelligence) [57]. that we rely on a random sampling process to draw solu- Most approaches to evaluating creativity have revolved tions, which already means that we perform some choice around the idea of separating between convergent and di- on the generative process. However, this generation being vergent thinking processes. The convergent thinking pro- an (hypothetically) entirely random decision mechanism, cess (a term coined by Joy Paul Guilford[31]) corresponds it can already be seen as a form of human choice on the to the use of knowledge and reasoning to solve a prob- creative process. Second, the flexibility aspect is related lem by eliciting a single solution. Hence, this delineates to the sampling of a pre-defined distribution, which lim- a corresponding set of questions that have a single cor- its the model to produce examples that remain consistent rect answer. Oppositely, the process of divergent thinking with its original observations. Therefore, this greatly lim- requires a framework where a wide variety of ideas can its the output of the model to the set of knowledge that be generated in response to a given question or stimulus. was selected and provided at the onset. These tests are usually derived from the seminal Torrance Test of Creative Thinking, which allows for the construc- As discussed earlier, expertise is a fundamental part of tion of associative hierarchies, such as asking to propose creativity, as it allows to obtain more efficient reasoning, different novel uses of a commonly known object [21]. based on appropriate problem representations, and recall- In the musical realm, another stream of research rely on ing domain-relevant patterns or invariant characteristics improvisation in jazz where a given melody should either [36]. Following our ideas from the previous sections, it be completed from memory (control) or freely (creative), also seems that this social context component in the eval- while keeping some parameters (length, tempo) fixed [43]. uation of creativity can be limiting to human creativity. Interestingly, this dichotomy would appear to provide a Oppositely, computational approaches can be freed from clear-cut and simple solution to our previous conundrum. these societal biases, and generate an almost infinite num- By crudely exaggerating, we could conclude that intelli- ber of solutions. However, this also introduces a novel gence is convergent, while creativity is divergent. How- caveat, that AI is trained to optimize a given distribution ever, it should also be noted that some criticisms exist on of pre-existing knowledge and it cannot evaluate the qual- the fact that creative processes also result from convergent ity of its own solution in any other way than the criterion, thinking [21], as creative ideas can be the fruit of labori- which is provided for training it. This lead to a paradox- ous trial-and-error works. Still, the domain of divergent ical situation, where there can be a boundless number of thinking seems to remain a privileged ground reserved to solutions, but in an inherently limited system bound by the creative behaviors. model, knowledge and criterion choice. All of these observations warrants the question of what 2.4.2. AI for creativity in the light of social sciences could be the place of AI in the creative process. Hence, we loop back to our previous question of finding a way to Following our previous discussions, it would first seem transition from artificial intelligence to artificial creativity. that AI approaches are only relevant to convergent think- Indeed, if we have access to a seemingly infinite genera- ing. Indeed, it is usually secluded to the definition of tion machine, but this machine is unable to evaluate its problem-solving approaches, where the learning is formal- own correctness, and only produces variations of existing ized by having a given single correct solution, which al- concepts, what creative use could we make of this system lows to understand and evaluate the quality of the pro- with peculiar characteristics. 3. ON THE INTRINSIC LIMITS OF AI FOR 3.2.1. Information SELF-CONTAINED CREATIVITY There is a widely agreed consensus that creativity is only We now try to cast a light on the limits of relying on AI to permitted through an existing body of knowledge, which produce its own self-contained creative behaviors, by dis- strongly influences the quality of creative outcomes. This cussing the main body of our argument, throughreviewing influence can be analyzed through the ways this knowl- all studies surrounding the creative process. edge is organized (information structures), retrieved (ac- cess strategies), analyzed (similarity evaluation) and even- tually transformed. 3.1. Studying the creative process 3.2.1.1. Organization The dominant paradigm to study the creative process scru- When receiving novel knowledge, humans create in- tinizes its structure as a set of stages, defining componen- formation structures and hierarchies allowing to memo- tial cognitive processes in a sequential or recursive fash- rize and organize it. At the neural level, it appears that ion. The seminal model of Wallas [58] separates the cre- neighboring neurons encode similar micro-features, and ative process between the preparation (information gath- distance between neurons can be interpreted as a proxy ering), definition (problem finding), incubation (reflecting for their feature similarity [26]. Furthermore, these dis- on ideas), illumination (appearance of a solution) and ver- tributed representations of micro-features induce a natural ification (testing the quality of the solution) stages. This modularity that do not require any proactive mechanism very linear model has since been more widely replaced for high-level organization. As the memory supposedly with a cyclical one, where stages are performed in various work in a content-addressable way, it appears reasonable combinations and are highly influenced by (extrinsic or that we gradually transitioned from coarse to finer repre- intrinsic) motivation [50]. Although it is now recognized sentations of memories [26]. Hence, we evolved towards that creativity is dynamic and interactive, looping between increasingly complex overlapping distributions, allowing different stages, the original components of Wallas remain for more interconnections and recall, leading to relation- widely used even in modern refinements. ships drawn from more integrative internal representation. In the view of cognitive theory, any phenomenon can These more complex associative hierarchies allow to potentially be recreated if we understand all of its princi- handle intricate concepts with a variable organization of ples. This should allow to address any complex and ar- cognitive elements, objects and relationships [32]. The duous problem by splitting it in smaller understandable appearance of these systems is theorized to have produced phenomena. The underlying hypothesis is that ill-defined emerging complex internal representation of abstract mean- problems can be broken into a set of smaller well-defined ings through symbols and their relationships. As proposed objectives [51]. This cognitive approach, which focuses by Deacon [19], we shifted from an iconic representation on the operators and strategies that come into play in the (simply storing physical or visual properties) to an index- creative processes has been a privileged playground for ical representation (representing a set of properties) and defining how AI should mimic creative behaviors. finally to a symbolic representation (where the representa- tion itself bears no similarity to the object it represents). 3.2. Mathematical reification of ideas For Deacon, the birth of symbols allowed us to imagine the use of objects as elements with correlations separate The abstraction of the creative process has been a core from their simple physical properties, enabling more intu- study for cognitive approaches and one of the most fertile itive and associative thinking processes. All of these ideas ground for the mathematical reification of ideas in AI re- are crystallized in AI around the representation learning search. In that view, AI should be bound to mimic human field [7], where the goal is to understand and learn proper- creative endeavors by reproducing the abstraction and tac- ties of a given set of objects in an unsupervised way (only tics that come into play in our brains. In that sense, the having access to the objects themselves). This is usually cognitive approaches have been mostly concerned with performed based on some more or less complex forms of fundamental processes that can be translated in compu- compression in order to produce higher-level and smaller tational terms. This entails the extent to which knowledge representation hierarchies. and information are organized and accessed, through dif- ferent types of memory systems, and the corresponding 3.2.1.2. Retrieval processes for retrieval and analysis of various sources of Parallel to the development of these specialized neural information. Subsequently, the operations applied to this circuits to structure knowledge, our brains also required knowledge allow to circumscribe the creative process in to define mechanisms to access and retrieve this informa- a computational sense. Here, we broadly separate these tion. Indeed, when engaging in creative thinking, we need views between these information and operation aspects, to rely on different granularities of representation levels, with some permeability across these categories. We study from very specific to highly abstract. The resulting re- the major sub-processes as they have been analyzed in trieval process obviously depends on the underlying orga- cognitive studies, while trying to tie links with how this nization but also the accessibility of different knowledge reflects on corresponding approaches in AI research. items [61]. The internal (physical) constraints of retrieval can provide either highly similar ideas, or oppositely aim tives [61]. This could be facilitated by modules of meta- to process abstract and distant concepts. representation, serving to represent "concepts over con- An encompassing model for understanding cognitive cepts" [54]. This interesting notion would permit to fa- information retrieval is the path-of-least-resistance pro- cilitate high-level reasoning across different domains, es- posed by Ward [59]. This model posits that our predom- pecially in the case of analogies and metaphors. Hence, inant retrieval mechanism is to access basic and specific this type of second-order system could provide us with low-level examples from a given domain as starting points, the ability to reflect on our own knowledge representa- while projecting their properties on the novel task at hand. tions and processes [20]. This suggests that we can use the representativeness of in- formation items as a retrieval likelihood function. It also supports the idea that remote associations, less represen- 3.2.2. Operations tative and distant concepts can lead to higher novelty [61]. Supplementary constraints such as latent inhibition allow Cognitive approaches emphasize the concept that individ- us to attend selectively to those information that appear uals generate creative ideas by exploring their knowledge through different types of operations. Hence, we discuss the most relevant, while screening out irrelevant knowl- edge. However, it has been shown that the lack of such in this section the ways in which we explore and develop inhibitions might provide a greater ideational fluency of relationships between different knowledge items. Several creativity [25]. Indeed, using more distant concepts have attempts to produce an encompassing model of such cre- propul- been shown to increase originality, but this may come at ative operations have been proposed, such as the sion model of Sternberg [55]. This model delineates a the cost of practicality. This process of contextually se- replication lecting some parts in information is also highly reflected classification of creative operations as (trans- redefinition in modern AI through the idea of attention mechanisms forming known ideas), (seeing known ideas in a new way), incrementation (extending known ideas), [56]. These approaches compute additional vectors that advance incrementation allow to contextually mask part of the information at dif- (similar but going further an ac- reconstruction ferent levels of processing, leading to the now well-known ceptable threshold), (reviving a previously reinitiation transformers models. abandoned idea) and (starting an idea at a rad- ically different new point). Although this proposal is in- teresting, we rather follow here a categorization which is 3.2.1.3. Analysis closer to the technical AI-based views and more prone Our ability for organized information retrieval allows to potential reification. Hence, we split different ideas to perform further analyses of this knowledge. This pro- between the modification (transforming a single item of cess has been scrutinized through the notion of contextual knowledge), association (linking several items or ideas), focus, where creative inspiration occurs when our atten- analogy (existing ideas are transferred to a new domain) tion is defocused, allowing more associative thoughts and and abstraction (finding a more general concept encom- to activate simultaneously distant representations [40]. This passing several existing ideas). form of defocused attention would allow to broadcast dif- fusely to broader region of our memory, allowing to con- 3.2.2.1. Modification sider a larger variety of elements rather than attentively The large majority of advances in any domain are based select distinct elements as in convergent thinking. This on small incremental changes in various ideas pertaining type of broad activation with looser definition of similarity to that field. These operations imply to slowly modify and selection could be primordial to associative thinking, existing ideas to include increasingly complex variations. while still being based on probabilistic relationships be- This appears logical as creative objects must usually strike tween knowledge elements. This posits the existence of a the right balance between familiarity and novelty. This be- separation between explicit cognition maps allowing con- havior can also clearly be seen in the publication patterns vergent reasoning for problem-specific approaches, while of any scientific field. This operation can also sometimes implicit cognition would allow to reason on distant cre- be evidencedeven within the productionof a single object. ative associations between elements [47]. Interestingly, For instance, in music, the notion of theme and variations these hypotheses contrast with our previous parallel to the is highly present, where the same material is exposed and attention mechanisms in AI, pointing out to a potential varied along a given musical piece. With this modifica- paradoxical limit in this reification process. tion operation, it is usually necessary to understand the existing similarity with the target idea and features inside 3.2.1.4. Transformation and meta-representation the given domain. This would imply a gradual access to Although transforming information could already seem the most readily similar instances given similar features, to belong to the operations category, reorganization and leading to a form of exemplar generation [60]. Although transformation might already occur on our existing infor- this can be seen in several approaches in AI, the variations mation structures. Indeed, as the phenomenon of neural remain in a confined set as delineated by the choice of plasticity is commonly accepted, there is also potentially the model and dataset, following our previous argument some reorganizing mechanisms to allow existing knowl- pointing out to this inherent limitation (Section 2.4.2). edge categories to be formed based on task-specific objec- 3.2.2.2. Association to reorganizing knowledge. However, these still do not The second type of well-studied creative operation is provide forms of higher abstraction. that of association or combination of existing knowledge and ideas. The conceptual combination implies to process 3.2.3. Limitations complex forms of similarities between apparently hetero- geneous ideas, in order to generate a novel concept by As we reflect back on our presentation of different aspects merging these elementary ideas. This notion of allowing of the creative process, we can see that our understanding two contradictory ideas to be entertained simultaneously, of different stages is limited to that of preparation (seen called Janusian thinking is critical to creative ideation. as information gathering) and incubation (seen as opera- Usually, this combination process allows the emergence tions on this knowledge). Following various components, of features that are beyond the simple summation of their there seems to be currently no equivalent to the steps of components. Hence, combining distant concept and rec- problem finding and illumination. However, these steps onciling their discrepancies allows to postulate novel prop- are critical to the creative process, and it appears that the erties that are absent from the original concepts, which is most functional parts of any purely creative behaviors are core to creativity [61]. Generally, the combination pro- still entirely human-based. This would point out to a form cess requires to perform integration between ideas that of partiality in artificial creativity, as the major aspects of are not usually grouped into a single coherent concept or its stages still seem to be out of reach. even space, which requires to break traditional thinking As we outlined throughout the definition of creativity patterns. Regarding AI approaches, this seems extremely (Section 2.1), the appreciation of its characteristics is usu- complex, as it transgresses the notion of physical proper- ally separated between the three factors of novelty, quality ties distance, and even requires to go beyond conceptual and social relevance. Hence, we try in the followingto de- distance aspects. To the best of our knowledge, this oper- lineate the potential limitations of AI for creativity based ation appears to currently be out-of-reach for AI models. on these three major criteria separately.

3.2.2.3. Analogy 3.3. Creativity and novelty Another type of process that has been repeatedly exam- The novelty aspect of creativity might be one of the most ined in creativity studies is the concept of analogy, where prominent in the collective subconscious. However, this existing well-structured knowledge is projected onto a novel might also be one of the most critical and complicated domain. On a simple basis, analogies can be used to apply point to address in a computational approach. This re- given solutions from a domain to another, or communi- lates to the way that systems are optimized by learning cating ideas in a more concise way. However, the real on a set of examples [30]. Indeed, training a given model force of analogical reasoning appears when it connects usually relies on minimizing the expectation of a loss (er- the source and target domains at very profound levels of ror) term. This implies that we are computing a mean er- knowledge, rather than merely on their surface [29]. As ror across a set of known examples. This also means that an interesting but somewhat looser parallel, this notion is the model is incentivized to perform correctly on the most currently highly studied and successful in the AI field, de- common elements of knowledge, by trying to fit the prin- fined as the task of domain transfer [9]. In this approach, cipal mode of the distribution (where most examples are part of the properties from an object (defined as source concentrated). Conversely, the unusual (outliers) exam- content) are transferred to a given domain (defined as tar- ples will have the lowest impact on the model training, get style). However, this process mostly aims to transform and will mostly be ignored to avoid skewing the global some low-level perceptual properties of objects rather than error term. However, as we discussed earlier, creativity some profound conceptual aspects. highly relates to dealing with these types of distant exam- ples. Hence, the major question is to know whether AI 3.2.2.4. Abstraction models could truly provoke novelty through some given The abstraction category is the most intricate and com- operational mechanisms, or if this would be bound to re- plex to grasp, as it involves to work at higher-level spaces quire a human intervention. Although AI is successful in of logical reasoning. Here, we link this idea to our inher- the organization and retrieval of knowledge, the combi- ent predictive system that is thought to be one of the criti- nation of conceptual elements drawn from memory stem cal function of the brain [21]. Evidence suggest that there from largely more complex processes. are general appraisal and reward-based mechanisms in the The psychologist Margaret Boden has given much at- brain that would provide incentives for learning. In that tention to the relations between creativity and machines sense, the abstraction operation could allow us to perform [12]. In this view, the ability to find new, surprising and more accurate predictions, by using these past rewards as socially valuable ideas can occur in three main ways: com- global indicators for enhancing our own decision mecha- binatorial (producing novel configurationsof familiar ma- nisms. This relates to our previous discussion on the def- terials), exploratory (discovering new paths in conceptual inition and learning of novel representation spaces (Sec- spaces) or transformative (when the space itself is dis- tion 3.2.1). Indeed, some approaches have tried to per- rupted giving way to ideas that were previously incon- form prediction directly in these spaces [42], as a proxy ceivable). Hence, there might be some yet uncovered op- erations that might favor creativity in AI, by empowering nicate, but also the way we think about concepts. Hence, their decision-making traits. This line of thought is princi- individuals with diverse backgrounds process information pally developed in the reinforcement learning approaches, differently, and have vast varieties in conceptual function- where agents are defined to explore spaces of possibili- ing [61]. These differences in language can be generalized ties. However, these require the definition of rewards and to their largest separation between taxonomic or thematic success functions that are still complex to define, as they ways of conceptualizing ideas [39]. The taxonomic func- relate to subtle perceptual and contextual aspects of the tions of languages is highly dominant in Western cultures, generated objects. where the objects and reasoning are "decontextualized". Hence, the relationships between objects are less impor- 3.4. Evaluating the creative quality tant than the category to which the object belongs to. Con- versely, the thematic language constructions which define Regarding the evaluation of the creative quality, there ap- Eastern languages put a larger emphasis on the context pears to be a strong duality in this question. On one hand, and logical or causal relationships between objects, rather learning approaches are trained by minimizing a criterion, than the individual object taken in isolation. Interestingly, which should serve as a proxy for quality. However, this this is also highly reminiscent and symmetrical to the the- loss usually pertains to some structural aspects of the gen- ological ramifications between Western and Eastern con- erated data, rather to more abstract aspects of its content. structs, which seem largely impacted by our logos. Hence, AI models are still unable to determine the real Hence, it might appear that the evaluation and novelty creative value of what they produce. To allow this com- of creativity remain entangled contextual-pronequestions. plexity of understanding, we need to be able to evaluate In orderto providean alternatepath to the use of AI in cre- both the creative product, but also the process itself. ativity, this requires to redefine the relationships between Hence, one of the major flaws of AI applied to creativ- human and machine in creative endeavors. ity appears to lie within its inability to judge the creative structures that emerge from exploratory processes. In a more global sense, this amounts to say that AI approaches 4. REDEFINING THE RELATIONSHIP WITH AI bear no artistic intent. Althoughthis appears to be a pretty THROUGH CO-CREATIVITY straightforward observation, this can be traced back to the question of measurement. Indeed, in order to provide a As we have seen across the previous sections, there are solution to this issue, we would need not only a computa- currently still some profound limitations to attain a true tional definition of creative ideas, but most importantly a form of artificial creativity. Indeed, current systems do not criterion of evaluation on what are low-probability events show any substantial musical creativity, lacking machine that still appear relevant. This question of relevance re- musicianship (the capacity to process music at a structural, lates strongly on the notion that creativity can only be symbolic level, a term coined by Robert Rowe [48]), high- evaluated within a given societal context. level interaction strategies or generative autonomy. How- ever, there still exists interesting avenues to truly harness the peculiarities of AI models in creativity. 3.5. The notion of context

As we emphasized in previous discussions, creativity is 4.1. Different views of AI in creativity highly contextual and deeply related to cultural aspects. One of the most notable differences in the conception of One crucial distinction needs to be made in the use of creativity can be found in the separation between West- AI for creativity depending on the role and place that it ern societies seeing creativity as a godly intervention con- is granted in the creative process. On the one hand, a veyed through an outside spirit, and Eastern societies that whole body of research appears to be devoted to use AI historically considered creation as a discovery or mimicry as self-sufficient generators, which seek to generate entire of something already pre-existing in the universe. This pieces of art. The system receives little input from the discrepancy in the evaluation of creativity can also be seen user, which now acts as a passive admirer of the creation. in what different cultures revere as a form of artistic ex- On the other hand, AI systems can be seen as creation pression. For instance, Japanese civilization highly values tools, which allows to enhance our own creativity. In that the works of calligraphers, sword makers and ceramicists regard, despite its higher complexity and intellectual ram- as art [52], which is less prominent in Western societies. ifications, AI simply holds a position akin to that of an This influence of culture on our daily life is so pervasive evolved brush to a painter. This second view provides en- and deeply intertwined with our reasoning that we are not hanced and facilitated ways to explore information spaces, even able to measure the true impact it may have. An but remain entirely subservient to our own creativity. emblematic example can be experienced in the excellent As we discussed earlier, when using AI in generative book Flatland by Edwin Abbott Abbott, showing that a processes, we still need to have a precise and well-defined person living in a two-dimensional world will most prob- problem, along with a set of representative data. Hence, ably never be able to reason about cubes. this already highly constrains the potential behavior of the Throughoutthese aspects, cultures are inherently linked algorithm, as we delineate the world of possibilities that to languages, as they represent both the way we commu- are attainable and explored by the model. Furthermore, we also implicitly limit the capacity of the models to eval- sounds synthesizer [1]. These types of spaces can pro- uate their own realization, as we define a certain train- vide an invaluable step in the organization of information ing criterion and loss function, which acts as a learning for the creative process, and can already serve as a gen- signal. However, this will also constrain the view of the erative tool to quickly explore high-level properties. An model to a single facet of the produced artefact. All of exciting direction of research would be to model more ac- these observations relates to the question of creativity as curately the geometric and topological features of these a problem-finding rather than problem-solving activity. It latent spaces across different learning settings. This ap- is sometimes more crucial to find an interesting question proach could provide insight on the learning process and rather to find solutions to existing ones. new mechanisms to perform unsupervised learning. Current models of computational creativity focus on These recent researches on information organization the ability to generate novel content. However, as dis- link to our question of expanding the human knowledge. cussed previously, the highly context-proneaspects of cre- Indeed, the strength of AI lies in its ability to understand ativity seem to confine this singularity-seeking approach multivariate and highly non-linear interactions in spaces to a pointless endeavor. Oppositely, it should be recog- with large number of dimensions. Hence, this aspect of nized that creativity traits should be shared between hu- knowledge structuring can enhance our own power over man and machines. Indeed, most works do not consider information structures. As a clear pragmatic example of the rich interplay and collaborations that emerge in the in- this idea, we recently introduced a radically novel formu- teraction and control of creative approaches themselves. lation of audio synthesizer control [24]. We formalized Indeed, most models are proposal generators, where we this as the general question of finding an invertible map- can pick from a list of ideas. Instead, we should aim to ping between organized latent spaces, linking the audio establish co-creativity partnerships focusing on the inter- space of a synthesizer’s capabilities to the space of its pa- actions dynamics, by evaluating a variety of collaboration rameters. This model allows to address extremely com- strategies. To do so, the first step is to define the problem plex tasks such as parameter inference (finding the pa- space itself, which can already be a daunting task. rameters of a synthesizer that reproduces any audio file), macro-control learning (disentangling and simplifying its 4.2. AI as a creativity-enhancing tool complex controls), and audio-based preset exploration (nav- igating sets of examples in a perceptually-relevant and Whatever type of AI we might define, it still requiressome intuitive manner) within a single model. This particu- form of human supervision. First, relying on AI as a lar instance shows that AI can be used as a mean to ef- creativity-enhancing tool requires to augment computa- ficiently re-organize intricate knowledge, even though it tional creativity models with artificial perception,by learn- is already accessible in a more entangled and harder-to- ing representation spaces for on-line music structure dis- harness sense. Hence, AI can be an extremely power- covery and generative decisions based on cognitive dy- ful creativity facilitator in its ability to organize complex namics by infusing social sciences in computing models. knowledge in a simplified way. Drawing an analogy to the previous section, first finding In order to decisively move towards artificial creativ- disentangled representation spaces of information is a de- ity, we critically need AI models that are able to listen and cisive and primordial aspect to any creative process. apprehend the musical structure correctly. These higher- This process of unsupervised learning is still poorly un- level cognitive phenomena involve long-term memoriza- derstood, even though it is one of the key towards new tion and structure discovery, usually performed on multi- generations of learning algorithms. These questions are ple time scales. In this direction, we worked on different crystallized around the field of disentangled representa- aspects allowing to perform online semantic information tion learning. The idea of representation learning is to extraction [13] and multi-scale musical structure predic- find compressed spaces (termed latent spaces) of infor- tion [14]. However, this also warrants the need to perform mation with a logical organization, while the disentangle- a machine evaluation of human creative behaviors in or- ment seeks to have the dimensions of these spaces to rep- der to understand and match the complexity dynamics of resent each factor of variation in the data separately. This musical interaction. This would allow artificial entities to question is usually approached through the use of Varia- infer new modes of a distribution in a self-supervised way tional Auto-Encoders (VAEs) [34]. Although the use of as a proxy for creativity. In that sense, an interdisciplinary VAEs for audio applications has only been scarcely inves- approach is mandatory, where social science and anthro- tigated, we recently proposed a perceptually-regularized pological studies need to collide with machine capacities VAE [23] that learns a space of audio signals aligned with and algorithmic approaches. perceptual ratings via a regularization loss. The resulting space exhibits an organization that is well aligned with 4.3. Towards models of co-creativity perception. Hence, this model appears as a valid candi- date to learn an organized audio space. Following this Overall, it seems that the limits of human creativity are research, we performed several follow-up studies to as- reciprocally limiting to AI, as we are bound to define pro- sess different generative aspects of these models, such as cesses that only mimic our own preconceptions of cre- adding musical conditioning[10], introducing other modal- ativity. Furthermore, there is a limit in the quest of self- ities of knowledge [15] or providing a continuous drum contained generative AI, as we require these approaches to generate novel content, but yet strongly conform to the dangerouspath of standardisation. While we will not dwell existing norms of that domain. A solution to this conun- into the questions of economic gain that could be inter- drum warrants the need to profoundlyredefine the creative twined with this prospect, we still underline the dangers relationship that may exist between human and machines. of indulging in these goals. The current supremacy of Indeed, instead of focusing on the either very subservient having highly conform thoughts and products, even in the view of AI as proposal tools, or oppositely seeing it as a artistic domain, can only lead to a dampening of creative self-contained generator, we should acknowledge that the endeavors. As an unconventional analogy from biology, true power to be harnessed comes from the partnership this would resemble a form of genetic drift in creativity between two separate systems (human or machine) with which should be avoided at all costs. This phenomenon each its specific characteristics. This idea of co-creativity is caused by continuously selecting a given subset of indi- emphasizes the fact that creativity is an emerging phe- viduals, which causes the reduction of genetic variance nomenon resulting from complex interactions and feed- and an overall impoverishment of the available genetic backs between actors involved in a creative process. This material. To transpose this in optimization terms, we would allows to regard not only the production of each (human constrain ourselvesto convergeand remain stuck in a local or artificial) part in isolation, but rather focus on the inter- minimum, by removing large parts of our search space. action that emerges between different components. Although these thoughts might appear as a bleak obser- Recently at the IRCAM STMS lab, Assayag [2] has vation on the use of AI for creativity, there are glimmersof thoroughly investigated artificial creativity issues involved hope hiding underneath. The advent of computational ap- in musician-machine interaction, especially in the case of proaches have propelled creative endeavors into horizons co-improvisation where agents of different nature (artifi- that could never have been reached before, and this path cial or human) perform together in highly unpredictable seems far from being exhausted. These exciting avenues live settings [5]. Assayag proposed the concept of hu- come from the fact that AI models and human behaviors man machine co-creativity in music [3], in order to over- can compensate for their respective limitations. In that come the aporia inherent to the essentialization of ma- sense, AI can easily process complex and multivariate in- chines when we try to assign them anthropocentric fea- formation without any prior bias. Hence, it can allow us tures such as creativity or intelligence, focusing then more to deal with objects and concepts that would easily exceed on the relations than on the intrinsic qualities of agents. our perception potential. This also question the episte- Co-creativity in that respect addresses emergent distributed mological reasoning of our quest to perform biologically- behaviors inherent to complex cross-learning feed-backs inspired mimicry, when we could harness the power of a between agents, and suggests reinforcement mechanisms truly novel and somewhat boundless generation tool. pertaining to co-action. This shift in objective allows us to question different ways to build the best possible tech- 5. CONCLUSION nical tools that could foster and enhance co-creative in- teractions. According to Assayag [4], co-creativity can Across this prospective paper, we tried to review the field only appear when two features linked to emergence and of creativity studies from a social and cognitive standpoint non-linear dynamics are identified: (1) emergence of co- and tie parallel links to the current development of AI hesive behaviors that are not reducible to, nor explainable models. Doing so, we aimed to provide a novel look at by the mere individual processes of agents; (2) appari- creativity in the era of artificial intelligence. After under- tion of non-linear regimes of structure formation, lead- lining the major limitations of current models from the ing to rich co-evolution of creative forms. The under- lens of cognitive studies, we discussed how the highly lying assumption is that such surging phenomena result contextual aspects of creativity and the question of qual- from cross-learning processes between agents involving ity measurements prove to be crippling to the singularity- complex feed-backs loops and reinforcement. As a ma- seeking approach. Hence, blindly following the aspiration jor consequence, the states and behavior of participating of self-contained generative machines, based on purely agents are in return modified continuously, making them mimicking human creative traits appear as a somewhat evolve in terms of knowledge and skills. These ideas in- pointless endeavor. Instead, we discussed the possibility volve intriguing prospect of modeling the dynamics of of redefining the relationships between human and ma- different actors that cooperate to perform complex adap- chines through co-creativity approaches. This allows to tation. This requires anticipatory systems of interaction, scrutinize and empower the link between both agents in with real-time adaptation that could account for the col- the creative process, rather than the agents taken in isola- lective dynamics in creativity. tion, as a main object of study.

4.4. Through the doors of perception 6. ACKNOWLEDGEMENTS As we discussed earlier, AI is inherently limited to opti- mize a mean accuracy on a given set of data, leading to This work is supported by the ANR MAKIMOno (17- the perilous situation of being an infinite norm-generating CE38-0015-01) project, the SSHRC ACTOR (895-2018- machine, which would be detrimental to creativity. This 1023) Partnership, Emergence(s)ACIDITEAM from Ville underlines the risk of overusing AI models towards the de Paris and ACIMO project from Sorbonne Université. References [15] A. Chemla-Romeu-Santos, S. Ntalampiras, P. Es- ling, G. Haus, and G. Assayag. Cross-modal vari- [1] C. Aouameur, P. Esling, and G. Hadjeres. Neu- ational inference for bijective signal-symbol transla- ral drum machine: An interactive system for real- tion. In Proceedings of the 22 nd International Con- time synthesis of drum sounds. arXiv preprint ference on Digital Audio Effects (DAFx-19), 2019. arXiv:1907.02637, 2019. [16] A. Craft. The limits to creativity in education: [2] G. Assayag. Improvising in creative symbolic in- Dilemmas for the educator. British journal of ed- teraction. In Mathemusical conversations: Mathe- ucational studies, 51(2):113–127, 2003. matics and Computation in Music Performance and Composition, pages 61–74. World Scientific, 2016. [17] D. Cropley and A. Cropley. Functional creativity. The Cambridge Handbook of Creativity, page 301, [3] G. Assayag. Human-machine co-creativity, a reflec- 2010. tion on symbolic indisciplines. In Artisticiel, Experi- ments in Cyber-Human Improvisations. RFI : Revue [18] M. Csikszentmihalyi. Society, culture, and person: francophone dâA˘ Zinformatique´ et musique, 2020. A systems view of creativity. In The systems model of creativity, pages 47–61. Springer, 1988. [4] G. Assayag. Reach project : Raising co-creativity in cyber-human musicianship. In Etincelle 20, La revue [19] T. W. Deacon. The symbolic species: The co- de la crÃl’ation. 2020. evolution of language and the brain. Number 202. WW Norton & Company, 1998. [5] G. Assayag, G. Bloch, A. Cont, and S. Dubnov. In- teraction with machine improvisation. In The Struc- [20] D. Dennett. Conditions of personhood. In What Is a ture of Style, pages 219–245. Springer, 2010. Person?, pages 145–167. Springer, 1988.

[6] R. Aunger. Darwinizing culture. The Status of [21] A. Dietrich. Brain mechanisms of creativity: What Memetics as a, 2000. we know, what we donâA˘ Zt.´ In Mobile Brain-Body Imaging and the Neuroscience of Art, Innovation [7] Y. Bengio, A. Courville, and P. Vincent. Representa- and Creativity, pages 23–28. Springer, 2019. tion learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intel- [22] J. Engell. The creative imagination: Enlightenment ligence, 35(8):1798–1828, 2013. to romanticism. 1982.

[8] C. M. Bishop. Pattern recognition and machine [23] P. Esling, A. Bitton, and A. Chemla-Romeu-Santos. learning. springer, 2006. Generative timbre spaces with variational audio synthesis. 21st International DaFX Conference, [9] A. Bitton, P. Esling, and A. Chemla-Romeu-Santos. arXiv:1805.08501, 2018. Modulated variational auto-encoders for many-to- many musical timbre transfer. arXiv preprint [24] P. Esling, N. Masuda, A. Bardet, R. Despres, and arXiv:1810.00222, 2018. A. Chemla-Romeu-Santos. Universal audio syn- thesizer control with normalizing flows. In 22nd [10] A. Bitton, P. Esling, A. Caillon, and M. Fouilleul. International Conference on Digital Audio Effects Assisted sound sample generation with musical (DaFX), 2019. conditioning in adversarial auto-encoders. arXiv preprint arXiv:1904.06215, 2019. [25] G. J. Feist. The function of personality in creativity. The Cambridge handbook of creativity, pages 113– [11] W. Blake. The marriage of heaven and hell, volume 130, 2010. 321. American Chemical Society, 1790. [26] L. Gabora and S. B. Kaufman. Evolutionary ap- [12] M. A. Boden. Creativity and artificial intelligence. proaches to creativity. The Cambridge handbook of Artificial intelligence, 103(1-2):347–356, 1998. creativity, pages 279–300, 2010.

[13] T. Carsault, J. Nika, and P. Esling. Using musical re- [27] H. Gardner. Creating minds: An anatomy of creativ- lationships between chord labels in automatic chord ity seen through the lives of freud. Einstein. Picasso. extraction tasks. In International Society for Music Stravinsky. Eliot. Graham, and Gandhi. New York: Information Retrieval Conference (ISMIR), 2018. Basic Books, 1993.

[14] T. Carsault, A. McLeod, P. Esling, J. Nika, E. Naka- [28] C. Geertz. The interpretation of cultures, volume mura, and K. Yoshii. Multi-step chord sequence 5019. Basic books, 1973. prediction based on aggregated multi-scale encoder- decoder networks. In IEEE 29th International Work- [29] D. Gentner, J. Holyoak, and N. Kokinov. The ana- shop on Machine Learning for Signal Processing logical mind: Perspectives from cognitive science. (MLSP), pages 1–6. IEEE, 2019. MIT press, 2001. [30] I. Goodfellow, Y. Bengio, and A. Courville. Deep [46] M. Puckette. The Theory and Techniques of Elec- learning. MIT press, 2016. tronic Music. World Scientific Publishing Company, 2007. [31] J. Guilford. Creativity. American psychologist, 5(9): 444–454, 1950. [47] A. S. Reber. Implicit learning and tacit knowledge. Journal of experimental psychology: General, 118 [32] A. B. Kaufman, S. A. Kornilov, A. S. Bristol, (3):219, 1989. M. Tan, and E. L. Grigorenko. The neurobiologi- cal foundation of creative cognition. The Cambridge [48] R. Rowe. Machine musicianship. MIT press, 2004. handbook of creativity, 216, 2010. [49] M.A.RuncoandR.S.Albert. Creativity research: A [33] J. C. Kaufman and R. J. Sternberg. The Cambridge historical view. Cambridge University Press, 2010. handbook of creativity. Cambridge University Press, 2010. [50] M. A. Runco and I. Chand. Cognition and creativ- ity. Educational psychology review, 7(3):243–267, [34] D. P. Kingma and M. Welling. Auto-encoding varia- 1995. tional bayes. arXiv preprint arXiv:1312.6114, 2013. [51] H. A. Simon et al. The sciences of the artificial. MIT [35] I. Kobyzev,S. Prince, and M. Brubaker. Normalizing Press Books, 1, 1996. flows: An introduction and review of current meth- ods. IEEE Transactions on Pattern Analysis and Ma- [52] D. K. Simonton. Foreign influence and national chine Intelligence, 2020. achievement: The impact of open milieus on japanese civilization. Journal of Personality and So- [36] A. Kozbelt, R. A. Beghetto, and M. A. Runco. The- cial Psychology, 72(1):86, 1997. ories of creativity. The Cambridge handbook of cre- ativity, 2:20–47, 2010. [53] D. K. Simonton. Origins of genius: Darwinian per- spectives on creativity [37] P.Locher. How does a visual artist create an artwork. . Oxford University Press, The Cambridge handbook of creativity, pages 131– 1999. 144, 2010. [54] D. Sperber. The modularity of thought and the epi- [38] T. Lubart. Cross-cultural perspectives on creativity. demiology of representations. Mapping the mind: The Cambridge handbook of creativity, pages 265– Domain specificity in cognition and culture, pages 278, 2010. 39–67, 1994. [39] E. M. Markman and J. E. Hutchinson. Children’s [55] R. J. Sternberg, J. C. Kaufman, J. E. Pretz, et al. The sensitivity to constraints on word meaning: Taxo- creativity conundrum: A propulsion model of kinds nomic versus thematic relations. Cognitive psychol- of creative contributions. Psychology Press, 2002. ogy, 16(1):1–27, 1984. [56] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, [40] C. Martindale. Biological bases of creativity. Hand- L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. book of creativity, 2:137–152, 1999. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008, [41] S. Moran. The roles of creativity in society. The 2017. Cambridge handbook of creativity, pages 74–90, 2010. [57] M. A. Wallach and N. Kogan. Modes of thinking in young children. 1965. [42] A. v. d. Oord, Y. Li, and O. Vinyals. Representation learning with contrastive predictive coding. arXiv [58] G. Wallas. The art of thought. 1926. preprint arXiv:1807.03748, 2018. [59] T. B. Ward. Structured imagination: The role of cat- [43] J. A. Plucker and M. C. Makel. Assessment of egory structure in exemplar generation. Cognitive creativity. The Cambridge handbook of creativity, psychology, 27(1):1–40, 1994. pages 48–73, 2010. [60] T. B. Ward. WhatâA˘ Zs´ old about new ideas. The [44] J. A. Plucker and M. A. Runco. The death of creativ- creative cognition approach, pages 157–178, 1995. ity measurement has been greatly exaggerated: Cur- rent issues, recent advances, and future directions in [61] T. B. Ward and Y. Kolomyts. Cognition and creativ- creativity assessment. Roeper Review, 21(1):36–39, ity. The Cambridge handbook of creativity, pages 1998. 93–112, 2010. [45] G. J. Puccio and M. D. Chimento. Implicit theories of creativity: Laypersons’ perceptions of the creativ- ity of adaptors and innovators. Perceptual and Motor Skills, 92(3):675–681, 2001.