Neurophenomenology

column 1 column 2 column 3 1 The Music of Consciousness: 1 2 2 3 3 4 Can Musical Form Harmonize 4 5 5 6 6 7 Phenomenology and the Brain? 7 8 8 9 Dan Lloyd • Trinity College, USA • dan.lloyd/at/trincoll.edu 9 10 10 11 > Context • Neurophenomenology lies at a rich intersection of neuroscience and lived human experience, as described 11 12 by phenomenology. As a new discipline, it is open to many new questions, methods, and proposals. > Problem • The 12 13 best available scientific ontology for neurophenomenology is based in dynamical systems. However, dynamical sys- 13 14 tems afford myriad strategies for organizing and representing neurodynamics, just as phenomenology presents an 14 15 array of aspects of experience to be captured. Here, the focus is on the pervasive experience of subjective time. There 15 16 is a need for concepts that describe synchronic (parallel) features of experience as well as diachronic (dynamic) struc- 16 17 tures of temporal objects. > Method • The paper includes an illustrative discussion of the role of temporality in the 17 18 construction of the awareness of objects, in the tradition of Husserl, James, and most of 20th century phenomenol- 18 19 ogy. Temporality illuminates desiderata for the dynamical concepts needed for experiment and explanation in neu- 19 20 rophenomenology. > Results • The structure of music – rather than language – is proposed as a source for descrip- 20 21 tive and explanatory concepts in a neurophenomenology that encompasses the pervasive experience of duration, 21 22 stability, passing time, and change. > Implications • The toolbox of cognitive musicology suddenly becomes available 22 23 for dynamical systems approaches to the neurophenomenology of subjective time. The paper includes an illustra- 23 24 tive empirical study of consonance and dissonance in application to an fMRI study of schizophrenia. Dissonance, in 24 25 a sense strongly analogous to its acoustic musical meaning, characterizes schizophrenia at all times, while emerg- 25 26 ing in healthy brains only during distracting and demanding tasks. > Constructivist content • Our experience of the 26 27 present is a continuous and elaborate construction of the retention of the immediate past and anticipation of the 27 28 immediate future. Musical concepts are almost entirely temporal and constructivist in this temporal sense – almost 28 29 every element of music is constructed from relations to non-present musical/temporal contexts. Musicology may 29 30 offer many new constructivist concepts and a way of thinking about the dynamical system that is the human brain. 30 31 > Key words • Neurophenomenology, music, ontology, temporality, fMRI, schizophrenia. 31 32 32 33 33 34 34

Philoso p hical- C ognitive once ts in N euro henomenology 35 Introduction: Time and the schoolbook story is to build a stable three this flowing filmstrip a percept emerges, in 35 36 consciousness dimensional model of the scene here and which we see the glimpses as a cup on a 36 330 37 now, a model in which various curves and table. Already at this point, time has made 37 38 It is a truism that time is important to shadows are parsed as a teacup, itself a stable its stealthy entrance. The multiple glances 38 39 human consciousness. But as with many concept without important temporal proper- are not simultaneous. To construct the 39 40 truisms, obviousness leads to a kind of ne- ties. Conscious life, according to this view, is a teacup, the succession of visual fragments 40 41 glect. In this case, one acknowledges time as series of such snapshots. Each is conceptually must be retained long enough to synthesize 41 42 a dimension of experience measured by the rich in itself, but in the normal case, where we an interpreted object in a coherent setting. 42 43 clock, while failing to recognize fully how are simply seeing the scene and recognizing This achievement is made more compli- 43 44 pervasive the experience of time actually is. the objects arrayed there, time is not particu- cated by the elementary consideration that 44 45 For example, consider an ordinary object, larly invoked. we observers are in continuous motion. As 45 46 a teacup. The teacup has a history and a fu- When we step outside of the school- our eyes make their saccadic leaps, we turn 46 47 ture, of course, but at first glance we might book scheme, however, we quickly observe our head, and possibly move in other ways. 47 48 suppose that we see it just as a momentary that real encounters with teacups are not Assembling a visual scene requires coordi- 48 49 presence, a three-dimensional form with- nearly so simple. To begin with the most nating the jumpy images with the equally 49 50 out noticed temporal properties. Our first obvious, our perceptual system cannot take jumpy traces of efferent commands to mus- 50 51 glance presupposition is tacitly abetted by in (and understand) a complex scene in a cles of the eye and the rest of the body. We 51 52 the schoolbook picture of visual perception, single glance. Observing a teacup, we fixate must somehow keep each image paired with 52 53 in which the eye works like a camera and its on the details of the handle, then the rim, concurrent position information and keep 53 54 images are translated and broadcast among then the saucer, then something the back- the parade in order, and from this build a 54 55 various brain areas. The job of perception in ground, then back to the handle again. From teacup. To the extent that I perceive a teacup 55 column 1 column 2 column 3

Constructivist Foundations vol. 8, N°3 Neurophenomenology The Music of Consciousness Dan Lloyd

column 1 column 2 column 3 1 here and now, and not a teacup fragment, I of immediate possibility he called Protention. The gears 1 2 need to hold in mind events in succession These flank the immediately given, which 2 3 that have occurred in the immediate past. Husserl called the “primal impression.” The Cognitive science circa 2013 draws on 3 4 In short, every teacup has a history that is Husserlian sandwich of retention, primal im- two broad paradigms for its explanations, 4 5 folded into its presence here and now, if it is pression, and protention characterizes every which comprise the tools for a science of 5 6 to be recognized as a cup at all. It has a future state of consciousness, regardless of inten- consciousness: dynamical systems theory 6 7 too, as is apparent if we should desire a sip of tional object. Importantly, the three facets of and computational modeling. “Dynami- 7 8 tea. Now we must anticipate the responses of temporality are co-present in the subjective cal system” and “computer” are each labels 8 9 the cup to our grasp and lift, a set of expec- now. In perceiving the teacup at 12:00:00, I broad enough to cover anything and every- 9 10 tations entailing a fluid grasp of the physics am also aware of its presentation at 11:59:59, thing, but as it happens the discourse of cog- 10 11 of our bodies and environment. How hard and its likely appearance at 12:00:01 – all of nitive science has tended toward somewhat 11 12 must the handle be squeezed? How much this packed into the experience at 12:00:00. more specific and concrete subtypes.N eural 12 13 upward force will be required to lift the cup? These phenomenological observations, networks have been the reigning dynamical 13 14 How must these forces be modulated to keep however elementary, set a high bar for scien- system, a natural choice for bridging brain 14 15 the tea inside the cup? What path will bring tific theories of consciousness. Consider, for and cognition. Computational theories 15 16 cup to lip? These, among other expectations, example, the binding problem, as presented have tended to stress language-like com- 16 17 shape how the cup is seen. Although these in a standard reference work: putational symbol systems, most notably 17 18 pasts and futures are not part of the sensory championed as the “language of thought” 18 19 field, they are among the contents of ordinary “ Information processing in the human brain is proposals of Fodor and others (Fodor 1975; 19 20 consciousness (Husserl 1974). They can be highly parallel. This means that different features Field 1978; Fodor 2008). Neither of these is 20 21 foregrounded through scenarios where the of an object are processed in different parts of the intended primarily to apply to conscious- 21 22 sensory presentation is fixed but the tempo- brain. For example, the color and the shape of a ness. Nonetheless, their different explana- 22 23 ral properties altered. For example, suppose red square are coded by different neurons in the tory powers immediately invite application 23 24 we discover that the teacup in question is a visual system (visual field). However, we do not in neurophenomenology. How do these two 24 25 very detailed hologram? The difference - be perceive ‘red’ and ‘square shaped’ separately but approaches fare in application to the prob- 25 26 tween a real teacup and a hologram is not the a ‘red square.’ The binding problem deals with lem of consciousness? In particular, how are 26 27 look of it but the clutch. As one’s hand passes the question of how features that are processed in they equipped to address the complexities of 27 28 unimpeded through the ghostly handle, the parallel are bound to the one unique percept.” subjective time? 28 29 contents of consciousness shift dramatically. (Herzog 2009: 388) To begin with language, decades of work 29 30 What shifts, however, is not the sensory pre- in linguistics and computation theory have 30 31 sentation but the expectations one holds. Temporality multiplies one red square illuminated the possible gears of the compu- 31 32 These observations of the ubiquitous into many, as even a static percept con- tational mind. We have the representational 32 33 presence of subjective time may perhaps tinuously updates a lengthening history. Its theory of mind and the language of thought 33 34 begin to hint at the phenomenological com- shifting retentions and protentions must in which the mind is a kind of software 34 35 plexity of ordinary states of awareness. Even also be bound together, but in a binding implemented in the neural networks of the 35 36 the simple cases in their most simplistic de- that preserves the structural subjective dis- brain. Can it stretch usefully toward neuro- 36 37 scriptions rapidly spiral towards Proustian tinctions between past, present, and future. phenomenology? 37 331 38 elaboration. (Note how different the teacup For every object of consciousness, in short, The powers of language are obvious 38 39 appears when you realize that it and its con- the brain must build a timeline, store it, and well-known. After all, the moderately 39 40 tents have been sitting on the desk overnight. and (crucially) keep it continuously pres- rich internal content of the teacup example 40 41 Cheers!) In 1907, Edmund Husserl devoted ent to mind as a non-sensory dimension of above is conveyed entirely in words, and I 41 42 an entire book to the simple encounter with every act of perception. Consciousness is a can reference it with just the phrase “the 42 43 objects (Thing and Space, Husserl 1974), in synchronic structure in which a formidable teacup example.” That is efficient, as well as 43 44 which pure phenomenology led to ideas that sea of diachronic information is represented vastly flexible (Did I mention that the tea- 44 45 would be repeated by phenomenologists (es- and continuously updated. cup is blue?). Syntax and pragmatics allow 45 46 pecially Merleau-Ponty 1962), and rediscov- Temporality is a feature of not just tea- endless molecular combinations of atomic 46 47 ered in James J. Gibson’s ecological psycholo- cups, of course. While we have not argued symbols, and of course time is included, 47 48 gy and in the recent philosophical fascination here that explaining temporality would be through tense and many other explicit 48 49 with embodiment (Gibson 1979; Clark 2003; sufficient for a scientific neurophenomenol- and implicit time markers. Syntax affords 49 50 Noë 2004). Husserl also opened the endur- ogy, temporality is certainly a prominent language much of its power, but seman- 50 51 ing theme of temporality in phenomenology and ubiquitous dimension of conscious life. tics lends a hand. For example, the atomic 51 52 (Husserl 1966); in this, he was influenced by Accordingly, the overall project of neuro- symbols (and hence their molecular expan- 52 53 William James (1890). The ubiquitous im- phenomenology is at least in large part the sions) are arbitrary. Nothing about the sym- 53 54 mediate comet tail of recollection he called project of neurotemporality. Time, as expe- bols, each taken in isolation, constrains the 54 55 “retention.” The equally ubiquitous branches rienced, will be a necessary component. reference or meaning that the symbol might 55 column 1 column 2 column 3

http://www.univie.ac.at/constructivism/journal/8/3/330.lloyd column 1 column 2 column 3 1 convey, once it is deployed as part of a sys- recurrent feedback loops in networks can sufficient in principle to account for a broad 1 2 tem of symbols. capture explicit temporal information as swath of cognition (and by extension, con- 2 3 While language can describe just about well (Elman 1990). sciousness); moreover, both are buttressed 3 4 anything (consciousness included), it is not These are very promising starting by sophisticated theoretical discussions of 4 5 so clear that conscious experience is itself points. However, between the simple mod- their capacities (with occasional nods to 5 6 a stream of symbols. (Extensive critiques els and the theoretical “in principle” dis- phenomenology, at least on the dynami- 6 7 include Dreyfus 1972; Varela, Thompson cussions lies a gulf. Simple models can be cal systems side). Between these low-level 7 8 & Rosch 1991; Wheeler 2005; Thomp- understood as functioning either via the demonstrations and the high-level theoreti- 8 9 son 2007.) Here, we can mention just the interaction of neuron-like computational cal proposals is a missing middle ground, 9 10 mismatch of serial symbols and synchro- units or as rudimentary functional divi- where our teacup is found. What is missing 10 11 nous temporal experience. The descriptive sions (layers or other discrete groupings are accounts of conscious capacities that 11 12 phrase, “the teacup example,” did not get of basic units). But tractable explanations approach the richness of human conscious- 12 13 filled with content in a stroke. Language is of real brains (and real consciousness) are ness without leaving behind its fundamen- 13 14 serial, and pearls of rich content are strung assumed to comprise functional archi- tal phenomenology 14 15 along with many less luminous beads. If we tectures and their dynamical interactions 15 16 sample the stream of description in the slim that leave units and layers behind. The as- 16 17 narrative of teatime, most of the symbols we sumed complexity is abbreviated in the idea The music of thought 17 18 find are denoting bare and local facets of the of high-dimensional state spaces, within 18 19 scene, if not still less. Consider any sentence which complex network states appear as To supplement dynamical systems and 19 20 word-by-word with special attention to the points. The concept of a high-dimensional (partly) displace the language of thought, it 20 21 content at each time point – what is usually state space partitioned into regions that can is proposed that neurophenomenology ap- 21 22 denoted is very limited. Conscious experi- be assigned to various cognitive states is an propriate the practical ontology of music. 22 23 ence, in contrast, seems to exceed this trick- appealing simplification of otherwise unin- Specifically, we now consider the possibility 23 24 le. Perhaps the language of thought admits telligible processes, but it remains vague in that the entities created and deployed by the 24 25 many parallel streams, but then the binding application. By what principles should the brain in the processes of consciousness it- 25 26 problem returns. There is no reason that the state space regions be demarcated? Are there self are analogous to musical entities. “Mu- 26 27 streams will bear any special relationship general rules that apply across tasks and sic” here is meant broadly as a set of system- 27 28 to one another. We could also broaden the modalities? I suggest that there is a miss- atic arrangements and constraints within 28 29 window of consideration, considering the ing level in dynamical systems theorizing, cultures for creating extended structures of 29 30 content of the text sentence-by-sentence something usefully situated between units sound. Music is thus the analogue of natural 30 31 rather than word-by-word. This still seems and layers on the one hand, and regions of language, and will inform theorizing much 31 32 to fall short of the phenomenology, but also state space on the other. This missing ele- as language does. Accordingly, we might 32 33 faces the very problem of temporality itself, ment is not a distinct process, but rather hypothesize a “music of thought.” In some 33 34 as a serial stream of tokens must be framed distinct entities – we need an enriched sci- ways, the music of thought hypothesis is a 34

Philoso p hical- C ognitive once ts in N euro henomenology 35 in a single synchronous representation. entific ontology to frame hypotheses about small revision in the language of thought 35 36 Synchronicity comes naturally in paral- the gears of conscious experience. Phenom- idea, but in others it makes a large and use- 36 332 37 lel distributed processing, the most popu- enology illuminates the features that this ful difference. 37 38 lar exemplar of dynamical system. Even intermediate layer should capture; here, we As a prelude, however, note that music 38 39 at the dawn of the neural network era, the have focused on temporality, for which syn- is not a frivolous sidebar to human evolu- 39 40 capacity for accommodating simultaneous chronous and sequential relations are es- tion. For at least 42,000 years humans have 40 41 constraints was celebrated (Rumelhart, Mc- sential. This desideratum distinguishes the been making musical instruments (Higham 41 42 Clelland et al. 1986). Early connectionist missing entities from language-like serial et al. 2012). Speech and song use the same 42 43 models demonstrated that networks of neu- symbol systems. However, the middle enti- anatomy, so it is reasonable to assume that 43 44 ron-like nodes could be adjusted (through ties should nonetheless capture the some or singing and talking mixed in the most an- 44 45 neural network learning) to solve reaching all of the generative (syntactic) properties cient social formations. Indeed, several 45 46 and grasping problems (Hinton 1984). In of language since the distinctions language researchers have proposed that music pre- 46 47 this case, a network maintained and up- draws are generally distinctions available to cedes language in human prehistory (Fro- 47 48 dated vectors whose elements represented awareness. ese, Ikegami & Beaton 2012). All known 48 49 some of the interplay of sensory informa- In short, both dynamical systems (such cultures have music, and most create and 49 50 tion evident in phenomenology. Neural as neural networks, typically) and linguistic share musical memes with enthusiasm. If 50 51 networks can be exquisitely sensitive to systems (typically understood as computa- one looks for a uniquely human production 51 52 their immediate antecedent conditions, in- tions over serial symbol strings) face simi- that could be the model of cognition, music 52 53 viting theoretical discussions of dynamical lar lacunae: both are illustrated through nu- shares the initial appeal of language. Indeed, 53 54 systems and time (van Gelder 1999; Varela merous simple examples and analogies that it may seem that music is just a subtype of 54 55 1999a). Concretely, models that include establish that each approach is plausible and language (Patel 2007). Most conspicuously, 55 column 1 column 2 column 3

Constructivist Foundations vol. 8, N°3 Neurophenomenology The Music of Consciousness Dan Lloyd

column 1 column 2 column 3 1 it shares a generative syntax (Lerdahl & which demands an infrastructure of tempo- respect to formalism and self-reference. 1 2 Jackendoff 1983). Like language syntax, rality and simultaneity. We can refer to their common ground as 2 3 musical syntax informs ordinary listeners Language affords a sharp distinction “internalism,” the view that musical syntax 3 4 (and informal musicians) with strong an- between syntax and semantics, a distinc- does not travel with a semantics of external, 4 5 ticipations of the next sounds in a piece tion absent in musical systems. Music does extra-musical reference. 5 6 and sensitizes listeners to violations. As not have language-like semantics. Occa- It was noted above that musical entities 6 7 with language, there are experts for whom sionally, music theorists have attempted are largely constituted through temporal 7 8 the syntax is explicit and invites theoreti- to specify how music can represent scenes relations, analogous to the constitution of 8 9 cal elaborations. But even non-experts are and actions, but these ideas are striking in states of awareness. Internalism, as defined 9 10 adept: just one French horn player who re- their failure. No orchestra can ever com- here, also has an intriguing analogy with 10 11 places a B with a B-flat will stand out in the municate “The cat is on the mat.” Musical phenomenology. Internalism as a phenom- 11 12 middle of a symphony, even to non-expert representation thus requires blurry denota- enological position would imply that a state 12 13 listeners. tions of indefinite conditions such as “feel- of consciousness is constituted through re- 13 14 Music has much in common with lan- ing” or “movement.” Even with this hedge, lations among other states of consciousness, 14 15 guage, but two conspicuous differences music seems to be unable to represent even and that the objects of awareness are mani- 15 16 make music an appealing metaphor for the simplest subject-predicate propositions. folds of states of consciousness, unified 16 17 consciousness. The first is that musical Even if a piece somehow depicts sadness, it through temporal relations. Phenomenolo- 17 18 symbol systems define most, if not all, of is unable to say who or what is sad (Kivy gy reveals the importance of temporality in 18 19 their symbols using time. The elements of a 2002; Scruton 2009). all awareness – at every moment, the teacup 19 20 musical system generally include pitch, tim- This sketchy semantics is wildly incon- of the example above is coordinated with 20 21 bre, harmony, rhythm, and melody. Pitch sistent with the highly specific syntax of its past and potential future appearances. 21 22 is arguably the atom of music from which all musical systems, which are exacting in Although there is a sensory presence (Hus- 22 23 the other structures are built. Pitch itself is their constraints on composition and per- serl’s “primal impression”), this is like the 23 24 dependent on frequency, a time-dependent formance. To some music theorists and phi- absolute identity of a chord. The edifice of 24 25 concept; but more importantly, the syntax losophers, this implies that music is nothing meaning attendant to the percept is entirely 25 26 governing well-formed symbol complexes but syntax, pure form without content (for contextual. With this discussion in mind, 26 27 in music is entirely temporal. Musical sys- example, Hanslick 1854, Scruton 1976). For Husserl’s metaphor for consciousness takes 27 28 tems include elaborate specifications of others, music carries rich content, but is en- on a new resonance. 28 29 simultaneous interactions among pitches tirely self-referential (e.g., Kivy 1990). Every 29 30 – these are the rules of harmony. Rhythm note in a piece represents other notes and “ When, for example, a melody sounds, the in- 30 31 and melody are governed by diachronic ultimately the whole piece and even the mu- dividual notes do not completely disappear when 31 32 laws. Duration defines the elemental beats sical tradition in which it falls. According the stimulus or the action of the nerve excited 32 33 of music, and sequences of durations spec- to this view, while musical symbols have an by them comes to an end. When the new note 33 34 ify rhythm. Sequences of pitches together absolute constitution of pitch and duration, sounds, the one just preceding it does not dis- 34 35 with their durations define melodies. The their interpreted meaning is always in ref- appear without a trace; otherwise, we should be 35 36 rules of harmony, then, define and interpret erence to their musical context. Every tone incapable of observing the relations between the 36 37 simultaneous interactions of rhythms and also specifies interval, harmony, tempo, me- notes which follow one another…. On the other 37 333 38 melodies. Finally, timbre, like harmony, is ter, and melody. Indeed, a core “utterance” hand, it is not merely a matter of presentations 38 39 specified by the variety of overtones of an in musical discourse is repetition, and its of the tones simply persisting in consciousness. 39 40 instrument and their temporal envelope primal modification is variation from rep- Were they to remain unmodified, then instead of 40 41 (Patel 2007). etition. Unlike language, musical repetition a melody we should have a chord of simultane- 41 42 Music thus falls intriguingly in the is never redundant. At every scale, every ous notes or rather a disharmonious jumble of 42 43 space between computation and dynamical element of a piece denotes by resemblance sounds…” (Husserl 1991: 30) 43 44 systems. Like a language system, a musical (or variation) the elements around it. The fa- 44 45 system comprises discrete atomic symbols mous four note motif that opens Beethoven’s The musical analogy may be especially 45 46 governed by a rich combinatorial syntax, Fifth is already a repetition-and-variation apt. 46 47 capturing the main appeal of the language structure. Its immediate repetition, one tone None of the above contradicts the im- 47 48 of thought hypothesis. Like a dynamical lower, is of course a reference to its first oc- portance of the reality of biological brains 48 49 system, musical entities emerge from simul- currence. And so it continues, ultimately re- in their ecological context. The music of 49 50 taneous parallel interactions. Musical syn- appearing in the third movement as a rhyth- thought does not sound in a vat, but rather 50 51 tax constrains these interactions, allowing mic motif on a single chord. A large part (if emerges from myriad internal and external 51 52 expressions that are both distributed and not all) of the artistry and pleasure of the causes. I suggest, however, that those causes 52 53 syntactically correct. In this respect, a mu- symphony rides on these symmetrical for- not be confused with content or the subjec- 53 54 sical structure and its dynamical possibili- mal interconnections. For present purposes, tive “feel” of experience (“what it is like,” 54 55 ties seems better suited to phenomenology, it is not necessary to take a position with Nagel 1974). 55 column 1 column 2 column 3

http://www.univie.ac.at/constructivism/journal/8/3/330.lloyd column 1 column 2 column 3 1 From metaphor to model But consonance is a feature of audible At this point, each component time se- 1 2 sounds, and whatever else we may claim ries was analyzed with the fast Fourier trans- 2 3 Music offers the prospect of a temporal about the brain, we know its myriad compu- form to yield a spectrum of frequencies for 3 4 grammar for dynamical states. The discus- tations unfold in silence. Nonetheless, the each, as if each oscillating component were 4 5 sion above highlights the fit of musical struc- analysis generalizes to any phenomena that a tone with a complex timbre. Each of these 5 6 ture with phenomenology. For the moment, can be analyzed as oscillations. Brains are signals can then be compared pair-wise to 6 7 however, this is merely a metaphor. Meta- included, particularly considering the al- all of the others, with a consonance (dis- 7 8 phors can usefully spotlight aspects of their pha, theta, and gamma oscillations detected sonance) value calculated for each pairing 8 9 targets, but for a science of consciousness we by EEG. Functional MRI yields time series (Sethares 2005). That is, each component 9 10 would like to reconstrue metaphorical fea- of images, and each pixel (“voxel” in fMRI tone (with its overtones) can be compared 10 11 tures as measureable observations. How will parlance) is an oscillator amenable to analy- to all the others, one by one, and the con- 11 12 this proceed? With a hypothesis as broad sis as a fundamental frequency plus many sonance measured for each two-tone chord. 12 13 as the music of thought, many avenues are overtones – although the sampling rate of Thus, among twenty independent compo- 13 14 open. For example, in another study I dem- fMRI affords oscillations at much lower nents there will be 190 unique pairs with an 14 15 onstrate that a statistical measure of sparse- frequencies than those observed with EEG. equal number of consonance measures. We 15 16 ness within symbol systems distinguishes These voxel time series may or may not be can calculate mean global dissonance for 16 17 natural languages from music (using several consonant with each other, using the same each subject, in effect taking the brain as a 17 18 hundred examples from several languages sense of consonance that applies to complex keyboard of “tones” and assessing its overall 18 19 and several world music traditions). In ef- tones. This will be a useful measurement if harmoniousness. This measure separates the 19 20 fect, sparseness measures a feature of syntax we can use it to draw distinctions between two groups (p < .01, two-tailed t-test). Mean 20 21 of these systems, which this study examined brains in different conditions. dissonance is greater in the patient group; 21 22 as single symbol tokens, dyads, and trip- Here, I will examine consonance as a brains affected by schizophrenia have more 22 23 lets. The sparseness measures used sharply global dimension of the brains of healthy tones and overtones whose frequencies in- 23 24 distinguished language from music. Then, subjects and schizophrenia patients. Data teract roughly. The sharpness of the divide is 24 25 the same analytic technique was applied to originally collected by Abigail Garrity et al. more apparent when the dissonance values 25 26 nearly one hundred fMRI image series from comprised 15 schizophrenia patients and 18 are not averaged. That is, we compare all of 26 27 three experimental groups to show that healthy controls performing the same audi- the pair-wise dissonance measurements for 27 28 these sequences of brain states were indis- tory oddball task: each subject listened to a all subjects. In this analysis, components 28 29 tinguishable from musical sequences (Lloyd sequence of tones, and pressed a button for from the brains of schizophrenia patients 29 30 2011a). a different “target” tone appearing at ran- differed from healthy controls with a signifi- 30 31 In this paper, I will examine another dom; the sequence was interrupted by odd cance of p < 0.000001. Overall, by both the 31 32 empirical application, taking up the musi- distracting noises (“surprises”), also occur- aggregate and mean measures, healthy sub- 32 33 cal property of consonance. Consonance ring at random. Each run was 372 seconds ject brains are more internally consonant 33 34 and dissonance are psychological percepts in length, with images acquired every 1.5 than the brains of schizophrenia patients. 34

Philoso p hical- C ognitive once ts in N euro henomenology 35 arising from the degree of interference be- seconds (for details, see Garrity et al. 2007). Do these differences reflect a steady 35 36 tween tones (Plomp & Levelt 1965; Sethares Images were preprocessed via standard state in the subject brains, or do the specific 36 334 37 2005). When frequencies are similar but not methods (with motion-correction, normal- components of the experimental task pro- 37 38 identical, their oscillations rapidly alternate ization to a standard anatomical template, voke different transient responses? We can 38 39 between constructive and destructive in- and spatial “smoothing” to reduce isolated assess this by calculating the instantaneous 39 40 terference. Over a certain critical interval, spikes of noise). Then, the image series were consonance of each subject through the ex- 40 41 this creates an audible beating that listen- further processed using independent com- periment, as follows. In the two calculations 41 42 ers experience as roughness. When a tone ponent analysis (ICA). ICA identifies tem- above, dissonance was calculated over the 42 43 includes overtones (as do most musical porally coherent networks, that is, sets of entire series of 248 images. But at each time 43 44 tones), then dissonance can arise between voxels that vary together over time, activat- point, component intensity varies. Thus, 44 45 neighboring overtones. Although this is a ing and deactivating in unison (Calhoun et if two mutually dissonant components are 45 46 perceptual feature of heard tones, like many al. 2002). In effect, ICA discovers networks both very active at a particular time, we can 46 47 musical features it correlates strongly with of correlated activity that can then be ana- consider the global dissonance of the system 47 48 physical properties of the sound. Thus there lyzed as “super-voxels.” A rather small set to be higher at that time. In this way, the 48 49 can be a straightforward physical calcula- of independent components (~20) gener- global magnitude of dissonance at any time 49 50 tion of consonance for any pair of complex ally captures 90% or more of the variance is modulated by the instantaneous intensity 50 51 signals. Physical consonance is not psycho- in the data. Here, ICA identified twenty of the more or less dissonant components. 51 52 logical (experiential) consonance, but their temporally coherent networks (brain re- Dissonance thus becomes a varying global 52 53 close psychophysical correlation warrants gions activating in unison) for each subject, parameter for each subject. 53 54 the application of the term as an objective and subsequent analysis was based on these With this continuous measurement, the 54 55 descriptor of frequency relationships. components. circumstances of sudden dissonance can be 55 column 1 column 2 column 3

Constructivist Foundations vol. 8, N°3 Neurophenomenology The Music of Consciousness Dan Lloyd

column 1 column 2 column 3 1 discerned. In healthy subjects, a transient structured, interdependent, temporal flow Conclusion 1 2 dissonance spike, compared to the baseline of consciousness. Language offers formal 2 3 levels of dissonance in each subject, oc- structures for characterizing cognition over- Music, like language, is culturally uni- 3 4 curs during two experimental events: at the all, but language does not offer an adequate versal, developmentally early, uniquely rich 4 5 sounding of a target tone, requiring a button model of the rich tapestry of conscious life. in humans, and evolutionarily adaptive (for 5 6 press; and at the random interruptions with In the earlier sections of this paper, it was ar- sexual selection and possibly communica- 6 7 a varying distracting noise. (The target dis- gued that musicology has developed analyti- tion, see Patel 2007). If language is embed- 7 8 sonance increase is significant (p < .001) as cal concepts appropriate to phenomenology, ded in our biology, then it is likely that mu- 8 9 is the increase in the presence of a distrac- especially in its temporal foundations. Con- sic is as well, possibly preceding language in 9 10 tor (p < .02).) The two events share several sonance is one such analytical concept. Al- evolutionary development (Froese, Ikegami 10 11 properties. At the most general level, they though it ultimately derives from listeners’ & Beaton 2012). Music shares with neural 11 12 are arousing and capture attention, and response to sound, like many other musical networks a capacity for parallel distributed 12 13 both require a decision to act (or withhold concepts it has a physical analogue, which representation, arguably a necessary condi- 13 14 action). In contrast, schizophrenia patients enables it to be applied to signals of all types, tion for consciousness. Musical systems are 14 15 show no significant difference in momen- including the varying activity of regions of as diverse as human culture, but share in 15 16 tary dissonance in response to any event in the brain, as detected by fMRI. Data from posing constraints on interactions among 16 17 the experiment. 33 subjects confirm that dissonance does musical elements. These constraints and the 17 18 If we consider the mean dissonance lev- indeed distinguish experimental conditions concepts used to describe them carve out a 18 19 els in the two groups, a possible interpreta- and subject groups. subspace of dynamical systems: musical dy- 19 20 tion emerges. Patient brain components are What then is the neurophenomenologi- namical systems. A musical dynamical sys- 20 21 consistently more dissonant than those of cal meaning of consonance and dissonance? tem thus has the ability to represent tempo- 21 22 healthy controls, and their level of disso- Consonance between signals increases when ral properties that are essential to full-blown 22 23 nance is statistically uniform for the entire their component frequencies are harmonic human consciousness. 23 24 experiment. In contrast, during target and (identical, or integer multiples of an implied Music, again like language, has its for- 24 25 distractor events, healthy subjects exhibit fundamental). Consonant signals are more mal methods of study. Its synchronic and 25 26 transient spikes of dissonance. That is, dur- often synchronized; they are more coher- diachronic form and temporal dynamics 26 27 ing arousing stimuli that capture attention, ent. Synchrony has been widely discussed have been described and modeled in cogni- 27 28 healthy brains become temporarily more as a mechanism for binding percepts (Engel tive musicology (e.g., Sethares 2005; Huron 28 29 like the brains of patients. Conversely, the & Singer 2001; Varela et al. 2001; Melloni et 2006; Sethares 2007). Although these for- 29 30 brains of patients are continuously like those al. 2007). Though the consonance discussed mal properties are based on sound, they are 30 31 of healthy subjects in aroused, attention-de- here is at low frequencies, it could be well readily adapted to the study of any complex 31 32 manding situations. suited to organizing experiences over sec- system. For any system, the fit of formal 32 33 onds and minutes as in Francisco Varela’s “1 musical properties can be probed empiri- 33 34 scale” (Varela 1999b). Interacting frequen- cally. Some systems will be “music-like,” and 34 35 The feeling of dissonance cies have also been proposed as a mechanism some will not – perhaps systems that are 35 36 for subjective awareness of duration, another “language-like” will exclude musical de- 36 37 As is generally the case in conscious- structural aspect of consciousness (Matell & scriptions. The cornucopia of musicology is 37 335 38 ness science, we have arrived at the mo- Meck 2004). Conversely, schizophrenia pa- waiting to be explored. In this paper, a single 38 39 ment of phenomenological interpretation: tients experience deficits in these capacities musical property showed potential as a tool 39 40 something in an experiment has produced (Lloyd 2011b), and other signs of disrupted for understanding consciousness in health 40 41 a measurable alteration in the brain. The temporality (Gallagher & Varela 2003). In- and schizophrenia, but this is only a tiny 41 42 experimental events are conscious experi- trusion by targets and distractors tempo- corner of an uncharted domain of explora- 42 43 ences. The brain alteration is thus correlated rarily desynchronizes brain components in tion. Music offers tools waiting for adoption 43 44 with the state of mind. This is the reason- situations that phenomenologically are also in the study of the conscious brain. 44 45 ing of “neural correlates of consciousness” intrusive and attention-demanding. The 45 46 (NCCs) (Chalmers 2000; Koch 2004). The healthy brain quickly reestablishes synchro- 46 47 present study is unusual, however, in that ny and returns to its baseline levels of con- Acknowledgments 47 48 it examines a global property of the brain, sonance. The conditions of arousal and ex- 48 49 and, moreover, that the property is derived ogenous distraction subside. Consciousness The author would like to thank Tom 49 50 by analogy from the analysis of musical continues, of course, but in the less aroused, Froese and an anonymous referee for the 50 51 sound. Both of these differentia are poten- “default” mode, with its mix of internally and journal, Constructivist Foundations, for 51 52 tially a good fit with phenomenology. Con- externally directed awareness. Disordered their extensive comments, and Susan Stuart, 52 53 sciousness science, being young, is open brains, in contrast, do not consistently estab- Bryony Pierce, and Mike Beaton for their 53 54 to new methods for characterizing global lish a similar degree of consonance. Schizo- generous editorial guidance throughout the 54 55 brain dynamics appropriate to a holistic, phrenia may be a disorder of harmony. writing and revision of this paper. 55 column 1 column 2 column 3

http://www.univie.ac.at/constructivism/journal/8/3/330.lloyd column 1 column 2 column 3 1 1 2 Dan Lloyd 2 3 { is the Thomas C. Brownell Professor of Philosophy at Trinity College, Connecticut. He is the 3 4 author of Radiant Cool: A Novel Theory of Consciousness (2004), a book joining noir fiction 4 5 with a theory of consciousness, and Simple Minds (1989). His current projects include Ghosts 5 6 in the Machine (forthcoming), a philosophical dialogue about minds, brains, and computers, 6 7 and Subjective Time (in press), an anthology on the philosophy, psychology, and neuroscience 7 8 of experienced temporality. His work at the intersection of brain and music is the subject of 8 9 a documentary in progress, The Music of the Hemispheres, directed by Elisa da Prato. 9 10 10 11 11 12 12 13 13 14 14 15 15 16 16 17 References Gallagher S. & Varela F. J. (2003) Redrawing the Husserl E. (1991) On the phenomenology of the 17 18 map and setting the time: Phenomenology consciousness of internal time (1893–1917). 18 19 Calhoun V. D., Adali T., Pearlson G. D., van Zijl and the cognitive sciences. Canadian Journal Translated by J. B. Brough. Kluwer, Dor- 19 20 P. C. & Pekar J. J. (2002) Independent com- of Philosophy 29: 93–132. drecht. 20 21 ponent analysis of fMRI data in the complex Garrity A. G., Pearlson G. D., McKiernan K., James W. (1890) The principles of psychology. H. 21 22 domain. Magnetic Resonance in Medicine Lloyd D., Kiehl K. A. & Calhoun V. D. Holt, New York. 22 23 48(1): 180–192. (2007) Aberrant “default mode” functional Kivy P. (1990) Music alone: Philosophical reflec- 23 24 Chalmers D. (2000) What is a neural correlate of connectivity in schizophrenia. American tions on the purely musical experience. 24 25 consciousness? In: Metzinger T. (ed.) Neural Journal of Psychiatry 164(3): 450–457.Gib- Cornell University Press, Ithaca NY. 25 26 correlates of consciousness: Empirical and son J. J. (1979) The ecological approach to Kivy P. (2002) Introduction to a philosophy of 26 27 conceptual questions. MIT Press: Cambridge visual perception. Houghton Mifflin, Boston. music. Clarendon Press, Oxford. 27 28 MA: 17–40. Hanslick E. (1854) Vom musikalisch Schönen Koch C. (2004) The quest for consciousness: A 28 29 Clark A. (2003) Natural-born cyborgs: Minds, [The Beautiful in Music]. R. Weigel, Leipzig. neurobiological approach. Roberts & Com- 29 30 technologies, and the future of human intel- Herzog M. (2009) Binding problem. In: Binder pany, Englewood CO. 30 31 ligence. Oxford University Press, Oxford. M., Hirokawa N. & Windhorst U. (eds.) Lerdahl F. & Jackendoff R. (1983) A generative 31 32 Dreyfus H. L. (1972) What computers can’t do: Encyclopedia of neuroscience 1 (A–C) theory of tonal music. MIT Press. Cam- 32 33 A critique of artificial reason. Harper and Springer, Berlin: 388–391. Available at http:// bridge MA. 33 34 Row, New York NY. www.springerreference.com/docs/html/ Lloyd D. (2011a) Mind as music. Frontiers in 34

Philoso p hical- C ognitive once ts in N euro henomenology 35 Elman J. (1990) Finding structure in time. Cog- chapterdbid/114175.html psychology 2: 63. 35 36 nitive Science 14: 179–211. Higham T., Basell L., Jacobi R., Wood R., Lloyd D. (2011b) Through a glass darkly: 36 336 37 Engel A. K. & Singer W. (2001) Temporal bind- Ramsey C. B. & Conard N. J. (2012) Testing Schizophrenia and functional brain imaging. 37 38 ing and the neural correlates of sensory models for the beginnings of the Aurigna- Philosophy, Psychiatry, and Psychology 38 39 awareness. Trends in Cognitive Sciences cian and the advent of figurative art and 18(4): 18. 39 40 5(1): 16–25. music: The radiocarbon chronology of Geis- Matell M. S. & Meck W. H. (2004) Cortico- 40 41 Field H. H. (1978) Mental representation. Erken- senklosterle. Journal of Human Evolution striatal circuits and interval timing: Coin- 41 42 ntnis 13(1): 9–61. 62(6): 664–676. cidence detection of oscillatory processes. 42 43 Fodor J. A. (1975) Language of thought. New Hinton G. E. (1984) Parallel computations for Cognitive Brain Research 21(2): 139–170. 43 44 York, Crowell. controlling an arm. The Journal of Motor Melloni L., Molina C., Pena M., Torres D., 44 45 Fodor J. A. (2008) LOT 2: The language of Behavior 16: 171–194. Singer W. & Rodriguez E. (2007) Synchroni- 45 46 thought revisited. Oxford University Press, Huron D. B. (2006) Sweet anticipation: Music zation of neural activity across cortical areas 46 47 Oxford. and the psychology of expectation. MIT correlates with conscious perception. The 47 48 Froese T., Ikegami T. & Beaton M. (2012) Non- Press, Cambridge MA. Journal of Neuroscience 27(11): 2858–2865. 48 49 human primates cannot decontextualize and Husserl E. (1966) Zur Phänomenologie des in- Merleau-Ponty M. (1962) Phenomenology of 49 50 objectify the actions of their conspecifics. neren Zeitbewusstseins [Phenomenology of perception. Routledge and Kegan Paul, 50 51 In: Scott-Phillips T. C., Tamariz M., Cartmill inner time consciousness]. Martinus Nijhoff, London. Originally published in French as: 51 52 E. A. & Hurford J. R. (eds.) The evolution of The Hague. Originally published in 1928. Merleau-Ponty M. (1945) Phénoménologie 52 53 language: Proceedings of the 9th interna- Husserl E. (1974) Ding und Raum [Thing and de la perception. Callimard, Paris. 53 54 tional conference (EvoLang9). World Scien- space]. Lectures of 1907. Martinus Hijhoff, Nagel T. (1974) What is it like to be a bat? Philo- 54 55 tific Publishing, Singapore: 126–133. The Hague. sophical Review 83: 435–450. 55 column 1 column 2 column 3

Constructivist Foundations vol. 8, N°3 Neurophenomenology The Music of Consciousness Dan Lloyd

column 1 column 2 column 3 1 Noë A. (2004) Action in perception. MIT Press, 1 2 Cambridge MA. 2 3 Patel A. D. (2007) Music, language, and the brain. 3 4 Oxford University Press, Oxford. 4 5 Plomp R. & Levelt W. J. (1965) Tonal conso- 5 6 nance and critical bandwidth. The Journal 6 7 of the Acoustical Society of America 38(4): 7 8 548–560. 8 9 Rumelhart D. E., McClelland J. L. & the PDP 9 10 research group (1986) Parallel distributed 10 11 processing: Explorations in the microstruc- 11 12 ture of cognition. Volume I. MIT Press, 12 13 Cambridge MA. 13 14 Scruton R. (1976) Representation in music. 14 15 Philosophy 51(197): 273–287. 15 16 Scruton R. (2009) Understanding music: Philoso- 16 17 phy and interpretation. Continuum, London. 17 18 Sethares W. A. (2005) Tuning, timbre, spectrum, 18 19 scale. Springer, Berlin. 19 20 Sethares W. A. (2007) Rhythm and transforms. 20 21 Springer, Berlin. 21 22 Thompson E. (2007) Mind in life: Biology, phe- 22 23 nomenology, and the sciences of mind. The 23 24 Belknap Press of Harvard University Press, 24 25 Cambridge MA. 25 26 van Gelder T. (1999) Wooden Iron? Husserlian 26 27 phenomenology meets cognitive science. In: 27 28 Petitot J., Varela F. J., Pachoud B. & Roy J.-M. 28 29 (eds.) Naturalizing phenomenology: Issues in 29 30 contemporary phenomenology and cognitive 30 31 science. Stanford University Press, Stanford 31 32 CA: 245–265. 32 33 Varela F. J. (1999a) Present-time consciousness. 33 34 Journal of Consciousness Studies 6(2–3): 34 35 111–140. 35 36 Varela F. J. (1999b) The specious present:A neu- 36 37 rophenomenology of time consciousness. In: 37 337 38 Petitot J., Varela F. J., Pachoud B. & Roy J.-M. 38 39 (eds.) Naturalizing phenomenology: Issues in 39 40 contemporary phenomenology and cognitive 40 41 science. Stanford University Press, Stanford 41 42 CA: 266–317. 42 43 Varela F. J., Lachaux J. P., Rodriguez E. & 43 44 Martinerie J. (2001) The brainweb: Phase 44 45 synchronization and large-scale integration. 45 46 Nature reviews. Neuroscience 2(4): 229–239. 46 47 Varela F. J., Thompson E. & Rosch E. (1991) The 47 48 embodied mind: Cognitive science and hu- 48 49 man experience. MIT Press, Cambridge MA. 49 50 Wheeler M. (2005) Reconstructing the cognitive 50 51 world: The next step. TheIT M Press, Cam- 51 52 bridge MA. 52 53 53 54 Received: 18 February 2013 54 55 Accepted: 27 June 2013 55 column 1 column 2 column 3

http://www.univie.ac.at/constructivism/journal/8/3/330.lloyd