August in San Diego: Neuroscience for Architecture, Urbanism & Design Michael A. Arbib University of California at San Diego NewSchool of Architecture and Design [email protected] (August 24, 2019) Table of Contents 1. Introduction + Evidence Based Design ...... 2 Neuroscience For Architecture, Urbanism & Design ...... 2 NfA and ANFA ...... 4 Evidence-based design ...... 5 2. From Neuroscience to Architecture and Back Again ...... 6 The Spectrum from Neuroscience to Architecture ...... 6 Core Neuroscience/Cognitive Science: The Action- Cycle and Affordances ...... 9 Embodied Cognition/Embrained Bodies ...... 10 Linking Music and Architecture ...... 11 3. Measuring Physiological and Neural Correlates...... 11 Core Neuroscience: The Autonomic Nervous System ...... 11 What can we measure and what does it mean? ...... 12 Architecture education ...... 12 Physical Reality “versus” Virtual Reality...... 13 4. Memory, Wayfinding and Design ...... 14 Core Neuroscience: Episodic Memory, Place Cells, and Navigation ...... 14 Wayfinding in and out of Virtual Reality ...... 15 Design – from Devices to Buildings ...... 16 Neuroscience of the design process ...... 17 5. Learning and Creativity ...... 18 Core Neuroscience: From Synaptic Plasticity to Deep Learning ...... 18 Artificial Intelligence, Creativity and Consciousness ...... 19 6. Empathy, Paradise, and Bio-Inspiration ...... 21 Core Neuroscience: Mirror Neurons and Systems ...... 21 Empathy, Einfühlung and Mirror Neurons ...... 22 Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 2

From Paradise to Courtyard ...... 22 Bio-inspired architecture ...... 24 7. Baukultur and Community ...... 25 What makes a good environment? ...... 25 Baukultur ...... 26 From Loneliness to Community ...... 27 8. Biology, Light, and Aesthetics ...... 28 Core Neuroscience: Circadian rhythms ...... 28 Light and Architecture ...... 29 Core Neuroscience: The Visual System ...... 29 What makes beauty? ...... 30 9. Neuromorphic and Dynamic Architecture ...... 33 Neuromorphic Architecture ...... 33 Core Neuroscience: Neuroscience of Emotion ...... 35 Emotions in a Neuromorphic Architecture ...... 36 Buildings as robots ...... 37 About the Author: ...... 38 References ...... 38

1. Introduction + Evidence Based Design NewSchool of Architecture & Design in San Diego held a highly successful Intersession on “Neuroscience For Architecture, Urbanism & Design” on August 12-15, 2019. Participants came from around the world, and many of the organizers and speakers were members of Academy of Neuroscience for Architecture’s Board of Directors or Advisory Council. To make sure that the excitement of the Intersession could be shared widely, I took copious notes and have now edited them, rearranged the material by themes that cut across the lectures (rather than offering lecture-by-lecture summaries), and augmented the result by adding brief introductions to core topics of neuroscience: the action-perception cycle and affordances; the autonomic nervous system; place cells, navigation and episodic memory; synaptic plasticity; mirror neurons; the visual system; circadian rhythms; and emotion. In addition, I have supplemented material from the Intersession with a range of personal observations that extend the discussion or offer new perspectives. This article is also being published as nine posts (one per section) on the AN blog on the ANFA website: http://anfarch.org/blogspot/. Neuroscience For Architecture, Urbanism & Design The “Neuroscience For Architecture, Urbanism & Design” Intersession held at NewSchool of Architecture & Design in San Diego on August 12-15, 2019 offered the following talks: Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 3

Monday, August 12 - Embodiment and Aesthetics In Search of Paradise | Harry Francis Mallgrave Neuroscience of Beauty | Tom Albright Neuroscience for Architectural Practice | Alison Whitelaw Light and Circadian Rhythm | Satchin Panda + Fred Marks Tuesday, August 13 - Human Centered Design Baukultur, Corbusier, and Buildings that have Brains | Michael Arbib User experience Design and, well, everything | Don Norman Designing for Active Healthy Cities at the Macro and Micro Levels | Jim Sallis Eco- and Neuro- Logical based Design for a post Anthropocentric era | Ilaria Mazzoleni Wednesday, August 14 - Technologies and Tools Measuring Neurophysiological Responses to the Built Environment | Eduardo Macagno Exploring the Spatial Relationships Between Real and Virtual Environments | Dane Clemenson Rethinking Architectural Methods with Neuroscientific Modalities | Biayna Bogosian + Kris Mun Responsive Architecture through Neuroscience-Based Material Programing | Elie Al-Chaer Mapping Trans-Disciplinary Processes | Myles Sciotto Emotive Matter: Neuroscience In Design | Behnaz Farahi Spatial Intelligence: Cyberphysical Architecture and Brain-Computer Interfaces | Guvenc Ozel Thursday, August 15 - and Perception Kinds of space | Sergei Gepshtein Enacting Civilization: Neuroaesthetics and the Thermodynamics of Beauty | Marcos Novak Home(ostasis): Towards an Architecture of the Extended Mind | Neil Leach The meeting was organized by Kris Mun, Tatiana Berger, Mike Stepner, Kurt Hunker and Elena Pacenti of NewSchool. Eleven of the speakers and organizers were members of the Board of Directors or the Advisory Council of ANFA, the Academy of Neuroscience for Architecture. The present document is based on the 40 pages of notes I took during the Intersession. Given that my typing for one subtopic might distract me during the beginning of the next, or that I might take time to jot down my own thoughts on the subtopic, the coverage of the lectures must be incomplete and at times inaccurate. Moreover, my own interests will have led to fuller notes on some topics than others for which I had less interest or understanding. Nonetheless, I hope that the edited notes will convey much of what was presented at NewSchool. As can be seen from the Table of Contents, this document does NOT offer talk-by-talk summaries in the order of presentation. Rather, in reviewing the notes, I sought to identify a variety of themes which, in some cases, cut across several talks, and then arrange them in an order which offers one possible path for cumulative understanding of the Intersession’s themes. Each section, then, is focused on just one or two of these themes so that readers can define their own path. The division of talks into topics and the division of the topics into individual Sections is thus somewhat arbitrary, designed both to provide one way of getting an integrated and Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 4 cumulative view of the Intersession while also keeping the length of top-level sections below 3000 words (some are half this length). Another feature of this document is that it emphasizes the neuroscience relevant to each theme, even though in many cases , cognitive science, or artificial intelligence may also be crucial. Thus all sections except this Introduction contain a subsection or two on Core Neuroscience. Specifically, these subsections introduce the action-perception cycle and affordances; the autonomic nervous system; place cells, navigation and episodic memory; synaptic plasticity; mirror neurons; the visual system; circadian rhythms; and emotion. I will rarely attribute “ownership” in the Core Neuroscience sections, but elsewhere, I will provide the surname of the person, e.g., Macagno, Mallgrave or Mun, on whose talk , or contribution to discussion, any material is based. In particular, I will use Arbib when summarizing my overt contributions at the meeting, but me or I or brackets [...] when my own observations or opinions need explicit labeling. Note that portions of the talk by any particular speaker may be distributed across several Sections. Q indicates a comment or question by a participant who was not a speaker or organizer. NfA and ANFA NfA denotes the discipline that ANFA was created to promote, but it has become clear that this must extend beyond “Neuroscience for Architecture” alone. Indeed, this blog is “the AN blog” to emphasize that the conversation goes both ways. To paraphrase JFK, “Ask not only what neuroscience can do for architecture, ask also what architecture can do for neuroscience.” However, this does not go far enough, in that each term may be taken in the narrow sense or in a broad sense. Architecture may be stretched to include, say, Interior Design and Urbanism; Neuroscience may at times expand to include Psychology and Cognitive Science on the one hand, and biological inspiration or neural net based artificial intelligence (AI) on the other. “The Spectrum from Neuroscience to Architecture” in this sense will be a core topic of Section 2 Alison Whitelaw gave us a hint of ANFA’s history, citing John Eberhard’s Brain Landscape (Eberhard, 2008) as the founding document, posing questions to neuroscientists concerning sustainable development – environmental, social and economic justice – with the built environment serving human well-being. Eberhard worked with Alison, Fred Marks, Gil Cooke and other architects to reach out to neuroscientists like Fred (Rusty) Gage, Tom Albright and Eduardo Macagno, leading to the foundation of ANFA. Considerations included sensory perception, the role of memory, the impact on affect and value. ANFA was created to promote scientific knowledge that may inform the design of the built environment. Architecture today is multi- disciplinary with a vast knowledge base. Technology provides tools, and neuroscience can create links from, e.g., synaptic details to health and well-being. At AIA 2003, Rusty Gage captured architect’s attention with his talk on neurogenesis. How can we add basic science protocols to architectural practice? It is hard to translate basic science into design. A roadblock has been the lack of funding for neuroscience studies on linking the built environment to enhancing the healthy human brain, though some large architectural firms are beginning to factor research in neuroscience (at least in the broad sense) into their planning. Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 5

Evidence-based design After her perspective on ANFA, Whitelaw focused on examples of evidence-based design that may in due course be illuminated by research in basic science and neuroscience, rather than on direct links between architecture and design. Perhaps the most cited study is “View through a window may influence recovery” (Ulrich, 1984) in which people with a view of nature through their hospital room windows recovered faster than those with a view of a brick wall. [I wonder whether wealthier people had access to better rooms?]. A PG&E study of schools showed that in classrooms with more natural daylight, students tended to have better test scores. Another study showed that stress responses can be reduced by access to good lighting in workplace settings. All this raises the challenge of factoring basic science into the design in a quest to understand the underlying physiological mechanisms. Dr Stanley Graven looked at neonatal ICUs, seeking to understand the differential effects of the environment on neonatal development. There is a huge leap in synaptic connectivity in the third trimester and early months after birth. First and second trimesters get structure in place with few connections. develop in a certain order, with audition and then vision maturing last. The sleep cycle starts at 28 weeks. Premature babies thus need ICUs that provide the necessary stimulation. Graven showed that single family rooms are beneficial, with lower sound and light levels. Kangaroo rearing (the baby spending time in a pouch against the parent’s chest) gave the baby more human contact that had beneficial effects including more sleeping time. The single room for ICUs are now established as preferred for hospital design. What might such studies imply for other aspects of hospital design? Or schools? Or ...? In kindergartens, less distraction, natural light, and children working at tables in small groups can all be beneficial. And then there are jails and prisons. The US has the highest rate of incarceration, with over 50% for drugs and immigration, offenses but only 2.5% for homicide and violent crime. Despite lots of data on the terrible side-effects of solitary confinement, some prisoners in Pelican Bay had been in solitary confinement for over 20 years -- yet the law did not realize mental harm, only physical harm. Perhaps a case could be made by seeking neural correlates – mental harm as physical harm to the brain, in the hippocampus and amygdala, and loss of visual ability and depth perception, orientation and wayfinding. Whitelaw has worked on a juvenile detention facility, a learning and research experience. Prefrontal cortex is not well developed till young adulthood. 62% of convicted juveniles had experienced abuse prior to incarceration, many with deleterious impacts on brain. A positive example was provided by the Missouri project that replaced juvenile halls by small cottages, more counseling, etc., with a marked reduction in recidivism. How can we design juvenile detention centers based on these insights, a trauma-based care model? Can one explore this scientifically without invoking inappropriate protocols? Empathy is a crucial part of this, developing role models and trust. We discuss mirror neurons and empathy in Section 6. Eduardo Macagno introduced the notion of Lifespan Architecture. Already an expert on neural development, he became interested in the neurobiology of aging when his mother suffered from Alzheimer’s disease and he found that a sense of space and place was one of the first skills to degrade. This motivated a concern with how space impacts people across the lifespan, and with strategies that can support varying requirements that change across the lifespan. Even for schools, a kindergarten student has very different needs from a college student. Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 6

Turning from neuroscience, Jim Sallis looked at evidence that lack of exercise impacts diabetes, stroke, mental health problem, etc. Lack of activity is particularly a problem of lower income societies. Sallis focused on beneficial elements of an active living neighborhood with a particular emphasis on the benefits of walking and biking – with a transportation system that requires people to walk or bike that last segment. 7 minutes per day of even slow walking adds up to 100 miles per year which, if one doesn’t eat more, can reduce weight by 3 lb./year. Public health is almost never factored into the design of public facilities and the neighborhoods and towns of which they are part. Active living research seeks to change this. On a macro level, cities can be designed to move people or to move cars. In fact, Manhattan is the most walkable city in the US. One secret is mixed-use neighborhoods, with multiple paths to vary walks between places. Freeways block connectivity for walkers. Sallis looks not only at the science literature but also data from “gray literature,” e.g., a transportation department evaluating effects of a new intersection or bike lane. This provides useful information that is not in the scientific literature. Conversely, scientists may get so hung up on details they lose the big picture. Lacking scientific studies of urban design and cycling, one may yet find cities with reports on relevant data that can be assessed to suggest useful innovations that are not scientifically proven. Among the comments made in the related discussion: If you need data for design, don’t ask a scientist ask a technologist. Their forte is basic research, not how best to apply it. But I still note the value of neuroscience, while stressing the difficulty of the multi-pronged conversation that has satisfactory design at the other end. Another comment noted the value of learning about the science: to provide inspiration not just information, helping reframe the problem. Basic insights may inspire the necessary reframing and later Sections will show the diversity of ways thinking about biology may inspire new questions and the differences this implies for design. One attendee spoke of learning so much here and from earlier meetings that he takes back to practice: designing the sound of opening a hotel door, placing a rose in a hotel bathroom to create a welcoming scent, providing the basis to help people tell stories about their experience. Another (little developed) discussion thread involved Architecture education. I will address this in Sec. 3. 2. From Neuroscience to Architecture and Back Again The Spectrum from Neuroscience to Architecture Tom Albright listed particular problems concerning architecture where neuroscience may be relevant:  Navigating through a complex space  Focusing attention on relevant aspects of a complex space  Assessing how the built environment influences social organization?  Effects of lighting on behavior and cognition  Learning, memory, communication in a developing child  What is beauty? [His views on this appear in the Section 8.] Architecture has always bowed to biology – whether designing for human dimensions, learning to decrease infectious diseases, or moving from door knobs to door handles. Albright Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 7 sees architecture as an applied science of human biology. Here he contrasts basic research with the process of invention that yields applications of science, distinguishing the three endeavors of scientific discovery, invention, and validation. Consider, e.g., using knowledge of biochemistry and genetics to invent new drugs that must then be validated by the FDA. To help people with memory problems, Albright worked with Sergei Gepshtein to invent signage that is context-, time -, and person-specific to jog memory. They now work with John Zeisel to validate this invention in homes for people with impaired memory. Zeisel is well-known for his pioneering work linking data from neuroscience to the design of such homes – see especially the neuroscience appendix linking hippocampus and memory in Inquiry by design: Environment/behavior/neuroscience in architecture, interiors, landscape, and planning (Zeisel, 2006). Science explores nature at many levels. Albright stresses that in any applied problem, one must seek a level of explanation that may support a causal mechanistic account that has predictive power relevant to that problem. For example, to explain airplane flight, invoke Bernoulli’s principle, not quantum mechanics. Similarly, for the many levels of neuroscience. To study navigation, study of ion channels may not be as helpful as higher-level analyses linking interaction of brain regions to synaptic plasticity. This relates to the observation that NfA must consider neuroscience in a broad sense – at times the appropriate level of explanation may lie within psychology or cognitive science rather than neuroscience. Harry Mallgrave noted that the Great Society programs of the 1960s aimed to eradicate poverty in a decade. However, the massive public housing projects they developed were abject failures. The problem may have been a failed sociological analysis – we need more research on patterns of community. Albright noted that Peter Barrett was asked to assess problems in the Manchester School system. Part of the problem was that two generations of people had been out of work. Thus the issue was not only one of classroom design and so forth – a major issue was how to get such people to become active parents, invested in the design and practice of the school, engendering their pride in the place. Q: Designing well-functioning cities is crucial, and mistakes are immensely expensive to correct. How might neuroscience help urbanists avoid some of these mistakes. Albright: It’s hard to do experiments on this. Juhani Pallasmaa stresses the importance of a humanist education to help architects better understand people – restoring cultural identity to architecture, linking science and humanism, not serving only developers. Maybe Virtual Reality (VR) could also be useful, in a different way. Note the CAVE at UCSD. One application helped divers navigate through rebar. But architects need to understand how people interact with each other in buildings. Eduardo Macagno stressed that neuroscience (including cognitive science) can identify principles of brain organization that may offer ideas for design and support evidence based design (as we saw in Whitelaw’s talk). His aim is to develop a toolkit for evaluating user experience Section 3. Conversely, he asks: What does architecture offer to neuroscience? He answers that it offers challenges for testing ideas on encoding space and place in our brains, noting Juhani Pallasmaa’s observation that spaces implicitly specify mental, emotional and physical states. In particular, he noted that “The Enactive Approach to Architectural Experience: A Neurophysiological Perspective on Embodiment, Motivation, and Affordances” (Jelić, Tieri, De Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 8

Matteis, Babiloni, & Vecchiato, 2016) emphasizes neuroscience as providing the biological basis for design in architecture, while architecture shapes human experience in ways that challenge neuroscience. Later Sections offer further ways in which architectural issues can motivate new research in neuroscience. Michael Arbib spelled out the difference between cognitive science and neuroscience of relevance to architecture: both seek mechanisms that explain the relation between experience and behavior, but only the latter invokes data that test hypotheses on the relation of those mechanisms to structures and functions of brains. Much work under the banner of “Neuroscience for architecture” is really “Cognitive science and evidence-based design for architecture.” As already noted, neuroscience experiments probe the brain at a multitude of levels from the social to the molecular. He also stressed that much can be learned about the human brain from an EvoDevoSocio approach: Exploring how biological evolution (Evo) yield brains and bodies that will develop (Devo) in an environment -- physical, built and social – that is itself the product in part of the social forces (Socio) that shape cultural evolution (Arbib, 2019a). He distinguished three challenges for NfA:  Neuroscience of the experience of architecture.  Neuroscience of the design of architecture  Neuromorphic architecture The first of these – as in the impact of a kindergarten lighting on children, or signage on Alzheimer’s patients – has been the primary focus of ANFA. Of course, the aim is to provide data that can inform the architect during design, but the issue of the brain processes engaged in the design process have received little attention, but see Section 4. The third topic explores the idea (already in its early stages of realization) that future buildings will have sensors, effectors and “brains,” see Section 9. Sergei Gepshtein organized his analysis around different notions of space. Finding C.P. Snow’s notion of a bridge between the two cultures of science and the humanities inadequate, he offered a spectrum from science to architecture bridged by diverse intermediary studies, enriched by a selection of books that he had found relevant. He sees the spectrum stretching from the mind-independent reality of physics to socially constructed reality. Physical space: Here he cites Max Jammer’s historical review of Concepts of Space. [All these concepts are socially constructed. In our Gifford Lectures The Construction of Reality (Arbib & Hesse, 1986), Mary Hesse and I sought to integrate her expertise in the history and philosophy of science, noting the role of the pragmatic criterion in social construction of a theory by a group of scientists, with my expertise in the ABC sciences of artificial intelligence, brain theory and cognitive science, assessing how each of us acquires the schemas that construct our experience of, and behavior in, the world around us.] Psychological space: Here Gepshtein showed Mach’s drawing of what he sees, including the side of his nose, to insert the human viewpoint. Psychophysics then links psychology back to ideas of physics. Books by Bela Julesz (1971) and David Marr (1982) were among those introducing further approaches to the mechanisms of vision. Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 9

Phenomenology: Sensory, motor and affective spaces are all different from “objective” physical space and this brings Gepshtein to Merleau-Ponty’s The World of Perception. He claims that this is far from the realm of science. Moving further along the spectrum he comes to instrumental space (he sees Linguistics fitting here) and poetic space. Citing Mikhail Bakhtin on the history of the novel and the notion of chronology he links this to narrative space, asserting that this “has nothing at all to do with physical space.” In every natural [or social?] situation, these very spaces are involved even if we focus on one or two at a time. He sees phenomenology at the break point between the scientific image of reality and the personal manifest image of reality. [One may compare the linkage of person-reality and ABC-reality in The Construction of Reality.] The Kanitza triangle is a familiar – three “pacmen” are arranged so that one sees the sides of a triangle even though no lines are there in the image. Physiological research may seek to explain the phenomenon of the apparent lines; bringing in physical accounts of perceptual organization, but for Gepshtein most architectural thinking is on the right side of the spectrum - ranging from phenomenology to narrative space and imagination. For architecture, he cites two books: Space in Japanese architecture (Inoue, 1985) defines space as space for movement [a view of the house akin to Lynch’s Image of the city (Lynch, 1960)?]. Dynamics of Architectural Form (Arnheim, 1977) yielded two relevant figures. One is a child’s view of a sequence of rooms, that captures the adjacencies in inhabiting the house rather than a Euclidean plan [the architect may combine both]; and a diagram by Portoghesi of forces of attraction and repulsion established by a building, invoking Gestalt theory’s metaphor of perceptual fields drawn from physics). [Portoghesi’s diagram reminded me of models of frog behavior in which the trajectory is shaped by a repulsive field representing a barrier and an attractive field representing the prey (Arbib & House, 1987).] Gepshtein then invoked Salvador Dali’s painting of his wife Gala linked to Harmon’s Lincoln image. From afar, we see Lincoln, near we see Gala, and there is a transition zone as we approach. The painting organizes the space around it. A good example. A scene in Venice has details apparent from afar while others can only be seen nearby. This led into exposition of a psychophysical study. As we vary contrast and spatial frequency of an array of bars, the higher the frequency the greater the contrast needed to distinguish the grating from a blank background of equal luminance (there is also a low spatial frequency effect). As we move through a structured space, spatial frequencies changes and so what we perceive changes - a form of narrative space. offers a form of narrative. Architects could design for this. [Indeed, in Gehry’s Guggenheim Museum in Bilbao, I was fascinated by the continual dynamics of the interplay between building structure and external views as I walked through the windowed spaces of the museum.] He thus distinguishes “the object” from “the phenomenology of an object as we move around a space” but I am not convinced that the gap between the two may be as large as he suggests. Core Neuroscience/Cognitive Science: The Action-Perception Cycle and Affordances Gepshtein and Albright both stress visual experience. Michael Arbib stressed that perception is action-oriented (Arbib, 1972) – it evolved in the first place to help creatures survive by sensing and responding to aspects of their environment relevant to their ongoing behavior. Perception is inherently predictive – the predator does not lunge at where the prey is now, but at where it will Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 10 be when his jaws close. The action-perception cycle (Neisser, 1976)emphasizes that, far from being simple stimulus-response creatures, what we perceive is guided by our ongoing plans, our actions may be directed to extend what we can perceive or to act to move towards our goals. And as we act, our perception of the world, our relation with the world, and the state of our environment may change accordingly, and so the cycle continues. Our and actions are guided by our “internal models” or schemas for the world. The cycle is thus a learning cycle, too – as we act and perceive we learn both when our expectations are met and when they fail. Neural plasticity supports learning and memory. In humans, our reliance on language and our interest in passive enjoyment of the arts may obscure the importance of action, leading us to ignore the EvoDevoSocio perspective– these are highly evolved and almost uniquely human capabilities. J.J. Gibson’s notion of affordances (Gibson, 1966) has become a major concept in these discussions. Gibson viewed “information pickup” as direct, denying the work of the brain, but he made the point that we may pick up information relevant to guiding action that may not need to enter our conscious awareness – as when we swerve to avoid a collision based on cues from peripheral vision before we are aware of who or what may collide with us, or we may duck when passing through a doorway whether or not we have made a conscious judgement about the height of the door. Thus the architect must in some sense have a narrative about who will do what in a building that is being designed, and factor appropriate affordances into the design accordingly. For humans, though, the aesthetic (in the sense, e.g., of appreciating beauty or atmosphere) complements the merely functional. Mallgrave noted that Gibson also distinguished habitat and niche. The former emphasizes where the animal or human lives, the latter is based on the way that an ecological niche impacts the biological of a group or species. The same notion is engaged in analyzing niche construction and its extension to cultural evolution (Iriki & Taoka, 2012; Laland, Odling-Smee, & Feldman, 2000). Embodied Cognition/Embrained Bodies Many speakers emphasized embodied cognition. The notion of the embodied mind emerged as a reaction to a privileging of cognition in separation from emotions and from the body. Certainly, architecture involves designing for the human embodied organism, whether our concern is with door handles, comfort, lighting or more. Embodiment is also relevant to understanding how the built environment socializes us, how we live in the environment ,as Harry Mallgrave does in considering the importance of mirror systems and embodied simulation in Section 6. However, I argue that the action-perception cycle is more fundamental: In architecture, the issue is not so much the embodiment per se as the range of actions to be performed and the affordances needed to support them. An aardvark’s brain in a human body would not be human; nor would a human brain in an aardvark’s body. Moreover, returning to the evolutionary perspective, our brains have evolved to go far beyond action, perception and embodiment in the here and now and to enter the realms of symbolism and abstract thought that may, but need not, have social significance. On the morning of my talk (at 6:20 am on August 13th, 2019) I had an epiphany on the distinction between the body and brain. In my earlier years at USC (in the late 1980s), my expertise in the neural mechanisms of motor control somehow led to my becoming a “guru” for the two departments of Physical Therapy and Occupational therapy. Physical therapy Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 11 traditionally focused on exercises designed to gain mastery over muscle movements, restoring or substituting “degrees of freedom” following a stroke or an accident or disease. Occupational therapy focused not so much on the muscles as such but on the tasks of everyday living, seeking to provide strategies that would be effective even if the effectors involved had to change. In each case, the focus was on manipulations of the body to improve function. My role was to help a transformation in practice and in research in which the findings of cognitive science and neuroscience could play a role, a transformation that has since been long embedded in the fabric of both departments. My epiphany was to realize for the first time that this aspect of my biography, one that had not entered my conscious mind for many years, was of special relevance to the discussion of how a subject like architecture could be enriched by neuroscience. The embrained and cognizant body. Linking Music and Architecture Many speakers reminded us that our perception is multi-modal – our experience of architecture may be dominated by vision, but audition and touch play crucial roles as well and may be seamlessly integrated within the action-perception cycle. Smell and even taste may affect our experience of a building or neighborhood. Myles Sciotto may have made a novel contribution to this in his talk associating music and architecture, in which he introduced a matrix for cross- modal integration between visual and auditory forms aimed at clarifying what is combined and related. Relations can be representational or actual, and may involve association, translation, or transformation. He noted the work of Xenakis as “musical metastasis” as in the blending of music and the building at the Centre Pompidou. Sciotto had determined that the piece of music written for the consecration of Brunelleschi’s Duomo could best be analyzed not in relation to the building itself, but rather to a drawing of the cupola that was transformed into the score. In one study, he scanned a Corinthian column, a relation of form and harmony, and took microphones and hooked them up to the scan. His convolution algorithm combined and blurred visual and auditory signals to let one hear what it looks like and see what it sounds like. [I was reminded of the dictum that “architecture is frozen music” and its more disturbing obverse that “Music is thawed architecture.”] However, his slides combined unreadably small font with diagrams whose complexity was left unexplained, and no audio was provided. I thus have no idea whether or not these methods can indeed yield fresh insights. Fortunately, readers can form their own opinion by consulting his UC Santa Barbara Ph.D. dissertation which “investigated and framed the field of Archimusic, the trans-disciplinary territory between architecture and music.” 3. Measuring Physiological and Neural Correlates Core Neuroscience: The Autonomic Nervous System Much of NfA stresses the role of the central nervous system in supporting action, perception, cognition, memory and more. However, Eduardo Macagno reminded us of the complementary role of the autonomic nervous system and its importance for homeostasis. It can control breathing, heart rate and many other bodily functions that complement the integration of sensory systems and skeletomuscular systems within the action-perception cycle. It contains two systems that in some sense work against each other. For example, the sympathetic nervous system constricts the pupil, the parasympathetic nervous system dilates it. Pupil dilation may be Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 12 measured both as a response to illumination level and as a correlate of stress/arousal. As a first approximation, the sympathetic nervous system can be associated with Fight and Flight modes, the parasympathetic with Feed and Breed. [Karl Pribram (1960) famously referred to these as the four F’s.] What can we measure and what does it mean? Much of our knowledge of neuroscience comes from animal studies, ranging from the genetics and molecular biology of basic cellular mechanisms including synaptic plasticity, via single and multi-cell recordings of neural activity during specific tasks, all the way to observations of behavior in single animals and groups both in the lab and in the wild. However, Eduardo Macagno focused on the challenges of measuring human response within the built environment. He noted that just putting an EEG headset on a subject does not mean that one can interpret the waveforms. Any experiment requires something like this methodology:  Define a question of interest.  Narrow it down to an answerable form.  Select the best methodology to gather data, e.g., using a questionnaire or physiological measurements.  Design the experimental paradigm and controls.  Do double blind analysis of data to avoid bias in the interpretation. Tools include brain scanning. fMRI offers good spatial resolution and poor temporal resolution of brain activity, but requires the subject to lie in a scanner with little or no movement. EEG offers poor spatial resolution and good temporal resolution and has the advantage of portability. Behavioral analysis can track overt movements, including eye movements, whether of the individual or group. An essay by Ann Sussman and Janice M. Ward1 in Common Edge argues that “Game-Changing Eye-Tracking Studies Reveal How We Actually See Architecture.” One may seek to study action in the built environment or evocation of certain emotions as a design goal. For the latter, pupil dilation and galvanic skin response may provide relevant data. Light wearable devices make measuring responses to the real environment more practicable. Studies may employ VR (virtual reality) but one then needs to study at least some of the functions in the real environment to at least establish how well the data may correlate. Different studies may require different measurements. To keep the costs of such studies under control, it is then preferable to network a few cheaper devices than use expensive commercial multi-purpose tools. To this end, Siddharth, Patel, Jung, and Sejnowski (2018) have developed a wearable multi-modal bio-sensing system capable of collecting, synchronizing, recording, and transmitting data from multiple bio-sensors. Such a system enables devices to “talk” to each other, unlike devices from different commercial vendors. Architecture education There was rather little discussion at the Intersession of how NfA could be factored into the Architecture curriculum. Should neuroscience be taught explicitly? Would architects benefit from more courses in the humanities? And so on. One speaker suggested focusing the last 2 years of an architecture degree on gaining a better understanding of the behavior and experiences of

1 https://commonedge.org/game-changing-eye-tracking-studies-reveal-how-we-actually-see-architecture/ Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 13 users of buildings. But “almost no architecture faculty have the necessary background for this and so we need to encourage students to go out and search for this knowledge themselves.” NfA progress requires that architects learn to pose questions that can indeed engage neuroscientists (in the broad sense). It may take a broad knowledge of the literature to find the relevant people with whom to engage on a specific problem, yet it is hard to get Architecture students to develop a skill for close reading of the literature. Don Norman stressed that science is creative and decries a hard boundary between art and science. His group at the Design Lab at UCSD is thinking of developing a Master’s degree that can build on an undergraduate degree that offers courses in the humanities and cultures. A weakness of the Lab at present is a lack of consideration of aesthetics. Intriguingly, when I asked him what principles of cognitive science should be part of the design curriculum, he could not specify any. He came up, rather, with rules-of-thumb and general strategies derived from design experience: For example, an ethnographic approach to design requires a good interviewing technique – and here it is important to learn how to listen. Indeed, when he joined Apple, Norman found that much of his knowledge of Cognitive Science was not directly applicable but, presumably, it informed his approach to solving design problems. For example, he used the term affordances in his book The Design of Everyday Things. Spending time in a science lab can be transformative in giving architects a sense of how scientists think. The most important contribution thus came from Eduardo Macagno. He observed that the graduate-level Certificate in Neuroscience for Architecture at NewSchool has three courses (one of which he has co-taught for many years) leading up to an NfA-influenced studio project, but NewSchool has no laboratory. With Tom Albright and others he sees laboratory experiments as the key to getting a feel as to what it is to do science, and neuroscience in particular. He has thus developed N-LEAD, the Neuro-Lab for Experiments in Architecture and Design as a possible model for expanding Architecture education when an NfA component is desired. With the help of Kris Mun from NewSchool, he set up a 4 week N-LEAD workshop at UCSD this summer (2019) bringing architecture students together with UCSD students in a mix of disciplines. Each day there were lectures in the morning by architects, device designers and others, with lab work in the afternoons. An important aim was affordability, keeping the budget for the devices used under $5K. Sample projects taught students how to use an eye tracker, a simple wearable EEG device, and a heart rate monitor. As a result, students learned a lot about how to design experiments and use devices. Physical Reality “versus” Virtual Reality. Kris Mun + Biayna Bogosian compare a scientific experimental process with an architectural design process. The latter may include interviews on the path to construction. A scientist might seek to quantify loneliness and, e.g., relate this to socioeconomic status, but the architect might relate it to square footage of housing and aspects of design. An overlap might look at the problems of lonely people in making eye contact. This led them to look at issues of visibility in relation to establishing connectivity. The study of air and sound quality in relation to stress could be a possible locus for work with neuroscientists, assessing foveal versus peripheral version, what vs where. They used isovist to understand visual connectivity in relation to San Diego’s airport, linking spatial organization to stress in moving through that environment. They also study how ambient sound affects stress levels at an intersection. Complementing field studies, they Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 14 structure spaces in VR to support experiments, accelerate design evaluation, allow sensory immersion, and combine biometric technology, real-time iteration, and even multiplayer experiments. Guvenc Ozel was asked about differential effects of physical reality, augmented reality (AR) and VR. He reported that he found head-mounted displays useless for experiences beyond placing a single object in AR. They don’t work for large scale spaces and environments. He also reported that it is hard to get VR displays that properly match the human visual system. VR is still in its infancy, but one can already design new experiences that people can immerse themselves in, despite their departure from normal patterns of experience. Dane Clemenson reported on an imaginative use of VR – using game players’ immersion in video games to study navigation. He had worked with Rusty Gage on environmental enrichment and its impact on hippocampus measured in animals, but now he seeks to use these insights in studying humans. He studies what we can learn from VR and how this relates to brain activity when navigating in the real world. His subjects play Minecraft, a video game which is built around complex open “worlds.” Success in such gaming requires good spatial awareness, as in navigating through a city. More on this in the Section 4. 4. Memory, Wayfinding and Design A major concern of this Section is with memory systems linked to specific brain networks, whereas the next looks more at general learning mechanisms at the synaptic level and their translation into artificial neural networks for Artificial Intelligence (AI). Core Neuroscience: Episodic Memory, Place Cells, and Navigation The hippocampus has long been a focus for study of brain mechanisms of memory. One key finding came from , the study of effects of brain lesions and neurological disease on human cognition and behavior. A young man named H.M. had such severe epilepsy that his surgeon decided (perhaps too drastically) to remove a massive bilateral area of brain including the hippocampus. Although banishing epilepsy, the result was drastic: HM lost episodic memory. Although he remembered events from before the surgery, he could not recall subsequent ones (Scoville & Milner, 1957). If you interacted with him, he might appear normal – his working memory seemed intact – but if your interaction were interrupted while you left the room, he would have no memory of you when you returned. The big surprise, though, was the discovery that H.M. nonetheless had procedural memory – he could acquire new skills. (Check out the movie Memento if you do not already know it.) If you played a new game with him on one day, he would not remember playing it the next. Any yet, after playing it again and again (each time believing it to be for the first time) he became increasingly skillful. These observations reinforce the points that (i) memory takes many forms, and (ii) that the brain is a collection of diverse regions making distinctive contributions to the patterns of activity and memory that cohere to make a self. We have already noted that John Zeisel has developed architectural strategies to help Alzheimer’s patients – and Alzheimer’s is a progressive disease whose initial impact on memory is associated with damage to the hippocampus. The other key finding about the hippocampus came from the use of single cell recordings from rat hippocampus as a rat navigated around a simple lab environment. O'Keefe and Dostrovsky (1971) discovered that certain cells fired not according to what the rat sensed or was doing but Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 15 rather as to which region in its cage or maze it was located. They referred to these as place cells. Building on this, O'Keefe and Nadel (1978) viewed The Hippocampus is a Cognitive Map while noting that there is also a taxon navigation system – we might paraphrase the latter by noting that we can navigate by referring to available affordances when we are in an environment for which we have no prior knowledge. My own group developed a computational model of these systems and their interaction, though we argued that the hippocampus does not of itself support a cognitive map but rather makes crucial contributions to an overall system for map-based navigation (Guazzelli, Corbacho, Bota, & Arbib, 1998). Wayfinding in and out of Virtual Reality Dane Clemenson uses Minecraft to study navigation and even creativity. The computer can explore moment by moment how people explore a rich Minecraft environment over multiple weeks, and he now has a setup to have people play Minecraft in fMRI (results are preliminary). He can also explore creativity by having subjects start with the same flat world in Minecraft and build a different world. All this raises the question of how well spatial ability translates between real and virtual environments. Using a videogame in which locations are based on Los Angeles, he studies whether there is transfer between real-world locations and experience of these locations as represented in a virtual world. [Certainly, many of us can do this by studying street maps to create an ad hoc cognitive map that guides actual navigation.] There was transfer between an object location task in VR or reality [RR: real reality?] with pre-exposure in the other mode, but only in some conditions not others. We may navigate space egocentrically following landmark-by-landmark directions for a specific route, or allocentrically, having a cognitive map that can support finding different routes with different destinations. To a first approximation, the former involves the striatum while the latter involves the hippocampus but, as already noted, these are embedded in a larger system. In an object location maze, Clemenson looked for transfer between navigating a virtual and an actual maze and found differences between use of egocentric versus allocentric strategies that could be assessed via analysis of errors -- but note that we get much more feedback when navigating the real world, in part due to proprioception. In summary, a range of studies explored the refined measurements of behavior that can be made in VR, while complementary studies demonstrated the extent of transfer between the VR and RR version of the tasks. One can learn from assessing this transfer. Another study, with Microsoft, studies active vs passive navigation. Relying on Google Maps does not yield a good cognitive map [as indeed is consistent with my group’s model cited above since using Google maps is a form of affordance-based navigation], and they are exploring the impact of using Microsoft Soundscape to generate auditory cues as to the direction to the next target. Q recalled the notion of aboriginal songlines. Aborigines can navigate across Australia by following a path associated with a song by repeating the words of the song which describe the location of landmarks, waterholes, and other natural phenomena. However, if not knowing the language in which the song was sung, “the melodic contour of the song describes the nature of the land over which the song passes. The rhythm is what is crucial to understanding the song. Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 16

Listening to the song of the land is the same as walking on this songline and observing the land.”2 Might this suggest ideas for infusing buildings and towns with auditory signatures? Design – from Devices to Buildings Don Norman lamented that there is unfortunately little interaction between Design and Architecture faculty in schools of architecture and design. He sees design as seeking a match between technology and people. Looking attractive is just one of these. His focus is on interaction design and user experience, emphasizing design in the small of devices and services rather than buildings. But perhaps these are not as disjoint as he suggests. For example, in designing a waiting line for Disneyworld rides, the solution was to add an anteroom to convey the story of the exhibit to distract from the fact that one is still waiting in line to get to the ride. In 1987, while still a cognitive scientist at UCSD, his group published a book User Centered System Design (UCSD!). After years in industry (including Apple) Norman returned to UCSD to found the UC San Diego Design Lab. The Lab takes a broad view far beyond the scope of Norman’s talk that extends to major societal issues such as large-scale education, automation, and healthcare. Design is viewed as a way of thinking “focusing upon how people interact with complex systems and technology, and upon ensuring that we solve the right problem, the root issues that define the true needs of the people and groups work with us, that we serve.”3 Clearly, such a system approach is relevant to the Urbanism thread of the Intersession, though this was outside the scope of the talk. Norman views science is a procedure for conducting repeatable experiments. Theories can guide, and are tested by, experiments – with incremental modification far more common than paradigm shift. Evidence may be hard or soft, but has to be replicable by independent people who do not share a common bias. In making devices, one can make prototypes and test them before going into mass production. Science requires precision, but for much design work we don’t need the same precision. Not what is the temperature, but is it comfortable? He noted that the Hong Kong Polytechnic has a new building designed by Zara Hadid. It has given the department great prestige — and yet it is a pain to work in. This is a conflict. How do we resolve the visual impact of prestige building versus people come first? Norman sees the Bauhaus as ignoring people, but I wonder if this is correct. Margarete Schütte-Lihotzky’s Frankfurt kitchen was a design to reduce the time spent in housework, so that women had more time for factory work or to pursue other interests. Among her influences was the small kitchen of the Bauhaus-designed Haus am Horn that had specific storage and drawers for specific items – a manifestation of the more general time-and-motion studies of Taylorism and a precursor of the workflow analysis that helps guide kitchen design today. A key observation was that with small devices design can be iterative: Build, test and modify. Norman is skeptical about human-centered design at the scale of architecture since one cannot iterate a building. He noted that after much iteration with unsatisfactory releases of Microsoft Window, Microsoft Windows 10 finally got it right. But I wonder whether this in some sense contradicts Norman’s point. Each release of Microsoft Windows was a massive system that was

2 https://en.wikipedia.org/wiki/Songline. Even though I grew up in Australia, I first learned about songlines from Bruce Chatwin’s 1987 book The Songlines, though the book has been criticized for its over-reliance on non-Aboriginal informants. 3 https://designlab.ucsd.edu/about/ Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 17 then tested by millions of users. Perhaps, then, the issue is not one of device size but rather the extent to which lessons learned from one form of a device or a single building or development can yield feedback that goes into the design of later devices or windows. Post Occupancy Evaluation (POE) is only carried out (according to one audience member) in 20% of major buildings. While this may or may not support upgrades to a specific building, it does add to growing expertise that is transferable to other projects. As Norman said, “Architecture is a social art, reflecting and changing culture” but it does not follow that “Architects design in a vacuum.” Experience and materials science research with glass yields new materials and new strategies for exploiting them in future projects. Moreover, research can be done as part of designing a building without resort to a full-scale prototype. At a talk in Taliesin West in 2011, Jeanne Gang discussed how her firm constructed scale models of protypes of the Aqua tower for Chicago along with the nearby buildings for wind tunnel testing that balanced the distinctive sculptural form of the balconies with the need to stop the winds of the Windy City making the balconies unusable. Today, designers of large buildings can call on experts in air conditioning (or its ecological avoidance) and “elevatoring” to combine science-informed engineering with cumulative architectural insights (and yet, see the next section, the result may be “new”). And Autodesk now has programs to prepare designs that architects can choose between to satisfy a set of specifications and constraints. Elena Pacenti observed that although we do test use, we don’t test social impact or its effect on the brain. She is intrigued by the rise of Uber and other ride-hailing services and stresses the need to understand its social impact at the level of urbanism – will they reduce traffic or not? – and the impact on taxi drivers both in the impact on memory and the downgrading of their profession. Norman commented on the impact of working at home, a long way from laboring in textile mills. The work shift is societal. Global business also changes work habits as people interact across time zones with very different 9 to 5. Meanwhile, office design has shifted from private space to open space and now may be swinging back again. He quoted the dictum of Herb Simon (a pioneer of Artificial Intelligence and a Nobel Laurate in Economics) that we are always predicting the future for that’s how we make decisions. This is, of course, a principle of animal behavior that guides our studies in cognitive neuroscience, not just for modern Organization (Wo)Man. Yogi Berra (the baseball philosopher) noted that prediction is hard, especially of the future. Neuroscience of the design process Michael Arbib considered the possibility for a neuroscience of the design of architecture by quoting a “challenge” from Peter Zumthor’s A Way of Looking at Things: “When I think about architecture, images come into my mind. When I design, I frequently find myself sinking into old, half-forgotten memories …. Yet, at the same time, I know that all is new and that there is no direct reference to a former work of architecture … “ In addressing this he recalled the linkage of hippocampus to episodic memory, noting that “There are striking similarities between remembering the past and imagining or simulating the future – and a common brain network underlies both memory and imagination,” quoting from “The Future of Memory: Remembering, Imagining, and the Brain” (Schacter et al., 2012). However, this quote is somewhat misleading since there are differences in brain activity between remembering the past and imagining the future, with different regions, as well as common Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 18 regions, involved in the two tasks. There are now a number of empirical relevant papers such as “Anterior hippocampus: the anatomy of perception, imagination and episodic memory” (Zeidman & Maguire, 2016). My own work (outside the scope of the talk) is addressed to creating a theoretical framework for addressing such data that builds on my work on How the Brain Got Language (Arbib, 2012b, 2016) yet links to the development of a neuroscience of design. The key idea is to see each of vision, episodic memory, and imagination as constructive processes. What we see is based on an assemblage of schemas that interpret different parts of the scene and the relations between them, but (as Tom Albright also emphasizes, see Section 8) this reflects top- down influences of our prior experience and current motivation and tasks (recall the action- perception cycle) as well as current sensory stimulation. Such an assemblage is not a mere labelling of regions of the scene; it captures aspects of shape, texture, location and more. An episodic memory is then a recoding of the assemblage, but the recollection of this memory will be only partial and in parts inaccurate, perhaps distorted by the task at hand – but it can again be based on a schema assemblage. Imagination then builds on diverse such memories, taking parts of different assemblages and distorting them as they are reassembled, with the schemas acting as dynamic entities to create new narratives – and these may be dreams unconstrained by current sensory input, or imaginations contributing to and constrained by an ongoing design process. The worked out details will be published in my forthcoming book When Brains Meet Buildings. 5. Learning and Creativity Clearly, creativity (this Section) and design (the previous Section) are overlapping topics, but where the previous Section centered on the functions of the hippocampus and its environs, the emphasis here is on synaptic plasticity and the way artificial neural networks (ANNs) have led to the current power of Artificial Intelligence (AI). Core Neuroscience: From Synaptic Plasticity to Deep Learning Neuroscientists have charted many brain mechanisms that support learning, but much emphasis has been placed on synaptic plasticity rules whereby the “weights” of the synapses (a synapse is a structure where one neuron acts upon another) change with experience. There are three main rules: In Hebbian learning, if both the presynaptic and postsynaptic neuron fire at the same time, the weight of the synapse between them will be increased. This was postulated by the neuropsychologist Donald Hebb (1949) and later shown to exist in the brain, first being identified in the hippocampus (Bliss & Gardner-Medwin, 1973). Supervised Learning was first developed in the Perceptron model by the psychologist Frank Rosenblatt Rosenblatt (1958), postulating that input synapses should be weakened if a neuron fired when a teacher said it should not have fired, and strengthened if the neuron failed to fire when it should not have fired. David Marr (1969) and Jim Albus (1971) treated the Purkinje cells (the output cells of cerebellar cortex) as Perceptrons, though Albus “got the sign right” – reversing the Perceptron rule because Purkinje cells are inhibitory. A breakthrough for machine learning (but a departure from neurobiology) came with the invention of backpropagation, an algorithm for “structural credit (or blame) assignment” that could be used to guide adjustment of synapses Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 19 for neurons in “hidden layers” (i.e., other than output neurons) in a feedforward network (Rumelhart, Hinton, & Williams, 1986). Reinforcement Learning does not require a teacher to specify correct firing of each cell in response to the current input pattern. Rather, a “critic” rates the overall behavior controlled by the network, and provides a positive or negative reinforcement signal to all neurons. A crucial extension was temporal difference learning that addressed temporal credit assignment in cases where reinforcement was intermittent (Sutton, 1988; Sutton & Barto, 1998). Although AI turned its back on learning techniques for several decades, one of the earliest great papers in AI anticipated temporal difference learning by developing a technique for machine learning while playing the game of checkers (draughts) where reinforcement comes only when one wins or loses the current game (Samuel, 1959). ANNs are simple abstractions from biological neural networks, augmented by varying “tricks” that augment the computations that support learning. Intriguingly, all these techniques were in place by the mid-1990s (Arbib, 1995) but their impact on AI remained limited. They can now do amazing things and drive AI thanks to their being able to work vastly faster than our biological neurons, the availability of immensely faster computers, and the ability to tap vast and reliable databases. We humans work with imperfection, but like other animals we have neurons of a diversity and complexity that are unmatched by ANNs, and we have a distinctive anatomy of brain regions, with distinctive neurons and connectivity that is related to that of other creatures (Kaas, 2017)– but with innovations that support unique abilities such as language. However, these neural and neurological subtleties have yet to factor into AI. Artificial Intelligence, Creativity and Consciousness Neil Leach brought the role of ANNs in AI – deep learning – into play by starting with the anecdote that, recently when boarding a plane in China, he did not need his boarding pass. Instead, an AI system recognized his face. The movie Blade Runner was released in 1982 but set in 2019. What did Ridley Scott get right and wrong? Replicants, no. Flying cars, no; but lots of drones. Dynamic facades, yes. And lots of AI. Deep learning had many successes that gained widespread attention: Deep Blue beat Kasparov; IBM Watson beat a human champion at Jeopardy; and AlphaGo won at Go. Can AI be creative? Can computers dream? A feedforward net can be trained with backpropagation to, for example, output BIRD with high accuracy when the input image contains a bird, but not otherwise. The Deep Dream study sought to infer a typical picture from the trained network by “running the weights backward” from the BIRD output. In this case, the picture contained lots of scrambled bird-like images. In a TED talk, Anil Seth built on this to discuss. “inside-out in perception,” while noting that too much yields hallucination. [This is similar to my “theory” of dreams but with interesting videos thrown in. Long ago, Richard Gregory (1967) argued that seeing machines will have illusions because economy of computation will require short cuts that usually work in the machine’s environment.] Another type of ANN is the generative adversarial network (GAN) in which two neural networks contest with each other, like an “artist” and a “critic.” Given a training set, a GAN can learn to generate new data with the same statistics as the training set. For example, after training Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 20 one GAN could create images that looked like human faces to human observers even though they were not photos of actual people. All this requires massive computing. This method has produced “This person does not exist” and “This Airbnb does not exist” sets of images. Shockingly, Portrait of Edmond Belamy, created by a GAN, sold for $432,500 on 25 October 2018 at Christie’s in New York. Wanyu He, CEO Xkool (Ex-Koolhaas and super kool) employs massive cloud computing and new algorithms to develop a database of buildings as a basis for generating new buildings. Currently she works only with 2D images. A database of Zara Hadid’s buildings can generate diverse novel Hadid-like forms. What happens as resolution increases, and the database goes to 3D? Is this the death of architecture? Well, rather than a team of architects, Leach imagines a lone architect with an app rapidly generating alternatives. Humans will increasingly interact with “packets” of AI as intelligent assistants [human-machine symbiosis], but will generally interact with apps rather than humanoid robots. In some sense, ANNs offer a Parametric Architecture with millions of parameters. In designing computer chips, engineers design for efficiency. Architects also consider beauty. [For some related notes, see (Arbib, 2019b), “Poetics and More in Performative Architecture: Towards a Neuroscience of Dynamic Experience and Design.”] With this background, Leach then turned to a more general discussion of creativity. Can AI be creative? Are architects creative? Going back to AlphaGo he noted that in the second game with world champion Lee Sedol, Move 37 was a move that humans had not made before. At first, it seemed mistaken, but it proved successful. Is this AI creativity? “AlphaGo showed anomalies and moves from a broader perspective which professional Go players described as looking like mistakes at the first sight but an intentional strategy in hindsight.”4 The University of Sussex cognitive scientist and philosopher Margaret Boden5 defines creativity as the ability to generate novel and useful ideas, distinguishing between historical creativity – it hasn’t been done before in history – and psychological creativity -- you haven’t done it before. It may be combinatorial, exploratory, or transformational. Leach speculated that maybe architecture is not that creative? To what extent does it follow a set of rules? Compare jazz as variation on a pre-existent theme. Is most architecture just variations on prior patterns? Compare Gehry’s Bilbao and LA Phil. There’s a signature there. Is deep learning so different from architectural training, viewing a huge range of images, with design as a variant of search on a vast latent space? [Recalling my riff on Zumthor, Section 4, I would suggest there is a distinction between minor variations and major innovations, even though both must build on prior experience. To my ear at least, some jazz players are more creative than others – though there is a fine line here between innovation and losing the audience.] To refute “the myth of originality,” Leach says Utzon just looked at the sails on Sydney harbor to come up with the idea for the Sydney Opera House. [But this is wrong, as I explain in an extended case study in When Brains Meet Buildings.] When will we stop calling AI artificial, Leach asks, and apply an adjective to us, labeling human intelligence as a special case? Compare our switch in terminology from “horseless carriages” to

4 https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol 5 I cannot resist mentioning that in her history of cognitive science (Boden, 2006), Maggie has a section on my work entitled “A Wizard from Oz.” Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 21

“cars.” We have here another Copernican revolution, moving beyond intelligence as human- centered. In many ways, Blade Runner was not about replicants so much as about what it means to be human. Marcos Novak commented that “if I take a robot to the gym it could lift the weights for me, but that would miss the whole point.” [Back to human-machine symbiosis. Getting the computer to aid our humanity. Cars let us travel faster, but they don’t destroy our interest in human movement and healthy activity. Asked in the 1960s whether AIs would take over the world from humans, Marvin Minsky replied “If we are lucky, they will keep us as pets.”] He also asked whether we would be more original if we did not use rules. Part of our creativity is developing new rules that can then allow new possibilities. Returning to Boden’s definition of creativity, to make something different is trivial. Even to make something useful is a limited form of creativity - a novel two bedroom apartment may not be well designed. Q. “What is consciousness. Do we attribute only human qualities to this? Can the machine have consciousness?” Gepshtein suggested that an enactive view (embodied cognition) removes consciousness from the brain and with that, there is no hard problem. Leach cited the view of David Chalmers that consciousness is a movie playing in our heads – but much of what we do is unconscious. But does this matter for machines? If a car driven by a human collides with a self- driving car, we ask about consciousness of the traffic of the former; for the latter the question is irrelevant.6 6. Empathy, Paradise, and Bio-Inspiration Core Neuroscience: Mirror Neurons and Systems Mirror neurons were first discovered as a subset of neurons related to manual actions in macaque premotor area F5 – they were distinguished by the fact that they were active both when the monkey performed a specific type of action and when it observed an other executing a similar action (di Pellegrino, Fadiga, Fogassi, Gallese, & Rizzolatti, 1992; Gallese, Fadiga, Fogassi, & Rizzolatti, 1996). By contrast, a canonical neuron in F5 fires when the monkey executes an action but not when observing it. In humans, we lack single-cell recording but can use brain imaging to identify a “mirror system” for a class X of actions or behaviors as a region that, compared to some control, is more active than other regions both when the subject performs an action from X or witnesses performances of an action of X. Mirror neurons are important, but we also need to assess brain systems “Beyond the Mirror.” There are regions active for execution but not observation of actions, and vice versa. In an imaging study, Buccino et al. (2004), had humans (i) watch three videoclips, one each for a human, a monkey, and a dog, biting food; they found the observer’s mirror system was activated in each case, whereas (ii) when observing a video clips (no sound) of a human speaking, a monkey making orofacial communicative gestures, and a dog barking, the mirror system for vocal communication was significantly active only in the first case. Buccino et al concluded that (a) actions belonging to the motor repertoire of the observer (e.g., biting and speaking) are mapped onto the observer's mirror system whereas (b) actions that do not belong

6 My views on how language makes human consciousness different from that of other animals were set forth in The Construction of Reality (Arbib & Hesse, 1986); see (Arbib, 2017) for the latest update. Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 22 to this repertoire (e.g., barking) are recognized without such mapping. My counter-hypothesis is that (b’) all actions are recognized by a system that complements a human’s mirror systems but that (a’) the mirror system adds “motor parameters” for actions in their own-repertoire Another imaging study (Wicker et al., 2003) showed that there is a mirror system for experiencing disgust and observing someone else expressing disgust. It is located in the anterior insula, a region very different from the premotor cortex area that houses the mirror system for manual action. But what I want to stress is that mirror neurons alone do not hold the key – the same study shows that many voxels are active for experiencing versus observing, and vice versa. Mirror neurons are assumed to be such that if an action is already in the animal’s repertoire, then they can aid recognition of similar actions when performed by another. But acquiring new actions through imitation requires more than this: if an action is not in the animal’s repertoire, and one observes another performing it as part of a behavior that yields a good outcome, then one can (possibly) learn to add it to one’s own repertoire as a basis for mastering the desirable behavior. As a final observation in favor of extending accounts of mirror neurons to include systems “beyond the mirror,” then, note that monkeys have mirror neurons but cannot imitate to any extent (let’s not spell out the details here), chimpanzees have “simple” manual imitation, and humans have complex vocal and manual imitation and language. All this provides challenges for the EvoDevoSocio approach (Arbib et al., 2018). Empathy, Einfühlung and Mirror Neurons Harry Mallgrave stressed the key insight that architecture is a social art. Recognizing that humans are profoundly social animals, we may consider both evolutionary history and what we share with other species, as well as our cultural and individual development, in thinking about architecture. Moreover, with Gadamer, note the role of ritual and festival for humans. This too needs to be factored into architectural design. In linking architectural theory, an intellectual framework, to neuroscience, he stresses the importance of 19th century German theory with its notion of Einfühlung, “feeling your way into a work of art or architecture.” He links this to empathy and its analysis in terms of mirror systems and embodied simulation. One may “feel” a work of figurative art through empathy of the persons represented there, simulating what is shown, but also through “feeling” the process whereby the artist created it. This goes beyond mental simulation of movement. Consider the broad brushstrokes of Kline, and the EEG study of the motor correlates of viewing such a work (Sbriscia-Fioretti, Berchio, Freedberg, Gallese, & Umiltà, 2013). Form is not neutral. Visual form evokes multi sensory and action-related effects. And space is not neutral. We measure space with our bodies. We shape our walking to the heights of doors and the width of corridors, but such changes also shape our perceptions. Space is not just an abstraction. From Paradise to Courtyard Harry Mallgrave quoted Alvar Aalto: “Each act of architecture aims to create a paradise.” Etymologically, paradise is a garden and gardens have been central to many cities across the centuries and across diverse cultures. But paradise can be approached in other ways, e.g., the acoustics of an abbey in which monks unite in Gregorian chant; or the use of stained glass in medieval European cathedrals. However, paradise is a different notion from utopia, in which new social structures are imposed from above. Mallgrave views paradise as finding the fit between environment and human nature, proceeding from within. As an example of town architecture Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 23 that exemplifies the idea of designing around gardens [or, at least, stretches of lawn] he cites the history of Oxford and Cambridge. They started by adapting monastic forms with buildings around a verdant courtyard, and then built more quadrangles, with about 40% green space. Later, bridges across the Cam gave the Cambridge colleges access to private gardens, and now the proportion of green space may be 80%. The town developed as a narrow section across the street from the colleges, providing services without intruding on the space of the colleges. There are also cultural amenities such as museums and music. However, the very success of these paradises has bred problems as the towns have become magnets for tourists, while industry and digital technology absorb the area around the centers. It is time to bring Aalto’s notion of paradise back into architecture! “What about underprivileged people? What about prisons?” someone asked. Mallgrave agreed that being a don or student at Oxford or Cambridge is highly selective but the question misses the point. They exemplify the urge for paradise — and he cites them because they can offer ideas for new urban design. Ilaria Mazzoleni cited several projects (see the next section), but the one most relevant here is PROTECT, exploring what gives us a sense of protection. How might early experience reduce fear and build memories that let us face nature or other aspects outside our usual environment? Connecting with nature, not just through the window, rather than raising kids with a phobia of the outside as in many cities today. Her ANFA 2016 talk focused on the courtyard, the making of a place enhancing biophilia. Many cities created buildings around large courtyards, but they often got infilled. Consistent with Mallgrave’s paradise, she advocated an architecture that embraces and opens to nature. Moreover, the courtyard can also be a place of social interaction. I will return to this in discussing Admiralty Kampung in Singapore in Section 7 Mallgrave takes a multi-sensory approach. He doesn’t downplay vision, but decries buildings whose designs seem based mainly on irrelevant visual effects, like the Gherkin and Walkie-Talkie buildings in London. Yes, we do need skyscrapers but – unlike London – Paris for example locates then outside the historical core. Similarly, he decries the trend in New York to taller and narrower buildings, but sees it as a result of immense land prices. He further decries the reflexive use of glass as a dominant material. One audience member noted the increasing use in New York of gardens and glass to give visual access as well as actual access. A good thing? Don Norman said that he finds some of the London buildings attractive but sees their assemblage as incoherent. He then suggested that some towns in Europe or Morocco are “incoherent in an attractive way.” He sees skyscrapers as offering social affordances - the urban density of New York City supports specialty stores and fine restaurants that could not be supported otherwise. Mike Stepner cited Temporary paradise? This 1974 study of San Diego by Kevin Lynch and Donald Appleyard lauded San Diego’s attributes that make it a paradise — the weather, the seaside setting – but they also but pointed out its shortcomings — traffic congestion, environmental degradation, and housing shortages, even then.7 Stepner, then city architect, tried to follow though, with a focus on truth and beauty. He noted the view of Roger Scruton – a

7 https://www.sandiegouniontribune.com/business/growth-development/sd-fi-temporaryparadise-20180326- story.html Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 24 philosopher recently reinstated as head of the U.K.’s Conservative government’s commission on “Building Better, Building Beautiful” – that beauty “is what it is all about,” but only if it is about people. Elena Pacenti followed up by commenting that San Diego has a beautiful environment but an urban design that is not walkable. To her, a walkable city (her home town of Milan) seems preferable to one with fine weather. Bio-inspired architecture In Section 9, we will consider neuromorphic architecture, buildings that have “brains” – but these may be based on principles from neuroscience or shaped more by the state of the art in AI. Here we look at other ways in which biology may inspire architecture. Ilaria Mazzoleni divides her year between Los Angeles and a valley in northern Italy. Her work embraces biomimetics, biomimicry, and E.O. Wilson’s notion of biophilia (and see neurobiophilia.org). With typhoons and earthquakes, we may see nature as an adversary; yet with woods and beaches we may hear the call of nature. She wants to move beyond anthropometric design. Certainly, architecture must continue to consider what is good for the human -- but one now needs to think this through in relation to the health of ecological systems and biodiversity. At ANFA 2016, Eric Kandel asserted that neural study for architecture is premature. [To my annoyance, he gave a talk on “art and science in Fin du Siècle Vienna” he had given many times before, having made no attempt to modify the talk for ANFA or think through the challenges involved prior to his visit.] Early AI experts rejected bio-inspiration [ironic, given the current dominance of bio-inspired deep learning] saying that airplanes don’t flap their wings, but Mazzoleni stresses that we are still looking to learn from flight in birds and insects. Her work links to biology but also has links to anthropology, ethnography and more – though one obstacle is that people from different disciplines uses the same word in different ways and may thus miscommunicate [e.g., contrast the use of “program” in architecture and computer science]. She introduced several projects beyond the PROTECT project noted above. Microbiome to microbi(h)one: A healthy person has a healthy gut and a healthy brain, so how might a house have bacteria that improve its performance as a home? Embracing bacteria for a healthier life, one may use bacteria for lighting while not over-sterilizing our lives InHabitLA_CoHousing (which will be at the Venice Biennale 2020): How can we live together and share facilities, open spaces, and biodiversity? CONNECT. There are now 9000 miles of sidewalks in LA. How can they be exploited to yield greater walkability? The sidewalk was invented in Pompeii, a small city, more stony than nature. City of Vernon in LA is a logistical center with only 109 humans – and yet there are many sidewalks! Why? This project assesses the conversion of miles of unused sidewalks into a linear park. Maybe different zones of a city need very different strategies, not just for sidewalks. Embrace pests, coexist with bugs, increase biodiversity, extending the human social connection to embrace other living forms. NAture-Art-Habitat Residency, NAHR: Neurological activity is affected by immersion in nature. Creativity is enhanced but in a different way. What (chemical?) changes in the brain does this support? Walking becomes a primary activity. New neural connections? Each year, a group spends a month in her valley in the Italian Alps, a place of nature and source of inspiration separated from the hurly-burly of city life and the internet. New ways of being alone and new Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 25 ways of being with others – even if the others are cows as well as humans. A dancer afraid of dancing on other than a flat floor ended up liberated by enjoying the challenge of dancing on the highly uneven surfaces of open fields. In mid-2019, the people of Ethiopia planted millions of trees. Small incremental changes by the individual can yield immense changes overall. One attendee had Indian ancestors who were animists, with respect for everything in nature. This attitude resonates with Mazzoleni’s projects. But do we want to return to an animist philosophy? Elie Al-Chaer spoke of biomimicry, learning from biological mechanisms that may have survived for thousands or millions of years. Such efforts may be structural, replicating form, functional, or ecological. However, biomimicry may be only skin-deep – Venhoeven’s Sportsplaza Mercator in Amsterdam has a leafy exterior but is not biological inside. A vision. Plant a seed from which a building grows. Buildings may be based on human bodily form, as in cruciform churches. The shape of the Blizzard Institute at Queen Mary University, London, is inspired by the shape of a neuron. Other inspiration comes from confocal microscopy, moving through the 3D forest of, e.g., the pyramidal neurons of cerebral cortex, or revealing the web of connections of neurons in retina. Another effort was inspired by study of cell membranes, with more positive ions outside, more negative ions inside. Channels with controllable gates control which ions to cross the lipid bilayer and when. Some gates are specific to sensory stimuli. The ear hair cells have mechanically-gated channels that convert sound energy into electrical signals. Depolarization at a synapse opens voltage-gated channels and calcium channels, with the latter crucial for neurotransmitter release. Ligand gated channels have specific keys for each type. All this inspired the architectural idea of a smart wall with dynamic channels, selective for different aspects to pass through. Let me note, though, that none of these is what I mean by neuromorphic architecture – perhaps confusingly I use the term for buildings that have “brains” in some sense, not for buildings whose form mimics a neural form. Another example of such an effort was given by Marcos Novak, who showed shapes that mimicked structures revealed in an image of his own brain. A famous non-example is Norman Foster’s Philological Library of the Frei Universität in Berlin whose floor plans are symmetrical in a manner reminiscent of horizontal cross-sections through the human brain’s cerebral hemispheres. However, we will see examples of neuromorphic architecture from Elie Al-Chaer in Section 9. 7. Baukultur and Community What makes a good environment? Harry Mallgrave commented that there has been much research on what makes bad environments, though understanding of high-crime or low-employment areas has led to little mitigation. But what makes a good environment? Can we learn from recent advances in understanding genetics and endocrinology and, say, the neuroscience of emotion? We need to understand the integration of body, mind and environment. We are complexly structured Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 26 dynamic, organismic systems. We must assess ,our integration into social and ecological systems. Each affects and is affected by our thinking and behavior. Architects must understand this, as we factor the built environment into the mix. Architects are cultivators of our world, constructing the niches which shape who we are. A walk in nature has been shown to benefit our health and raise our mood. But we cannot go back to primeval nature in our habitat. I would add that our appreciation of nature is culturally specific – Australians today greatly appreciate the distinctive beauty of their landscapes, yet early English settlers could only “see” what was reminiscent of the English countryside. How might we bring essential aspects of nature into our built environment? Looking at architecture solely in aesthetic terms leads us to neglect the impact on human well-being. We do not just use a room, we inhabit it. Walking down a street of uniform facades can bore us, and create stress. Eduardo Macagno presented two images and asked, “What feelings do these images evoke?” One was a mountain stream, nature. He stressed that we don’t just register the 2D image. Rather, it evokes a multimodal sense of space and affordances. The second image was of the interior of the Cathedral of St John the Divine in New York City. Macagno found it to be a haven, a place for de-stressing, when he lived in New York. For him, the image evokes a sense of calm contemplation. Mallgrave also commented on the positive effect of the space within a cathedral, so “design with nature” is not the full answer to a healthy environment. The “nature” in which many people live is highly urbanized, and apartments may be cramped. He contrasted a Hong Kong “micro-apartment “with a San Diego “macro-apartment,” noting that only the well-off can afford the latter. But whatever the size of our home, our sense of space and place are informed by memories of prior sensorimotor interactions, affected by culture and age. Good design enriches the private world of the individual experiencing it, says Mallgrave – so a homeless person might value a micro-apartment while others would not Guvenc Ozel believes that architecture is about creating habitations that detach us from nature. With VR, one can seek to create an infinite new space that can be used productively. Architects need to be engaged in this. Gaming experiences tend to be designed by engineers, not professional designers. But what are the ethical issues as we erase nature? Are we destroying the natural in favor of the machine? Don’t we need to keep the separation between the imaginary and the real? Currently, that process of detachment from nature is proceeding at an increasingly fast pace, and that is causing problems. Art may refer back to nature or not, but it is not nature. The only ethical question is whether the way we live artificially will support or destroy our continued existence. Baukultur Michael Arbib discussed the Davos Declaration, “Toward a high-quality Baukultur for Europe” (Davos, 2018a, 2018b), in which Baukultur (building culture) combines a society’s particular culture of building (how they go about creating their built environments) and the building of this culture (how the quality of what they do and what they produce can be raised). He argued that Baukultur in this sense is, indeed, a global challenge. Complementing this, Colin Ellard (2018) stressed implications of Baukultur for “urban planning [that takes] seriously the implications of built design for human mental health.” Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 27

A society’s culture is dynamic. When the Eiffel Tower was built it was widely denigrated, yet it is now (along with the recently burned Notre-Dame) a beloved icon of Paris. We seem ill- prepared to predict when innovation will destroy the culture of a city (as Mallgrave believes the skyscrapers are doing to London) or will come to be seen as making a vital contribution to the city’s evolving culture. Moreover, as Arbib illustrated with a photo of the new mosque in Cologne with the cathedral in the background, a given city or region may include diverse cultures. Is a mosque a symbol of cultural breadth or a stimulus for hatred in traditionally Christian cities? What can architecture – as distinct from education, moral leadership and politics – do to advance the harmonious mix of diverse cultures in a harmonious whole? To move forward on this issue, we may need to explore for Baukultur the EU principle of subsidiarity. The OED defines this as “the quality of being subsidiary; specifically the principle that a central authority should have a subsidiary function, performing only those tasks which cannot be performed effectively at a more immediate or local level.” Might this notion provide a framework for the Baukultur of rapidly changing, multicultural societies? As a link to neuromorphic architecture (Section 9), Arbib noted that the Davos Context Document states that “The fourth industrial revolution has begun. The vision of interconnectedness between virtual and physical devices in a global network, the ‘Internet of Things’ is becoming fact.” In response to the Davos concern that “cultural values, such as the authenticity and historic originality of material and substance, may lose importance in favour of more standardised images and perceptions,” he noted that modern technology can indeed support culturally distinct devices. Japanese examples included the electrical kotatsu and the futon “hair dryer.” Of course, the ubiquity of skyscrapers raises concern that architecture may become more homogenous. From Loneliness to Community Kris Mun + Biayna Bogosian noted problems with Le Corbusier’s vision of the city, and the failure of mass housing in Chicago and St Louis. They bemoaned the repetitive undifferentiated sameness of the new cities of China. And with this, they noted that there is an epidemic of loneliness around the world, and more so in cities than rural communities. The college age cohort is the loneliest recorded, and San Diego has high levels of loneliness. UCLA studies show that loneliness can change the brain and body in deleterious ways. They currently study ways to address personalizing high-density housing in San Diego, placing an emphasis on the human. What can NfA offer for a healthy coliving typology? [Recall related comments by Mazzoleni.] A book on the Ladakh depicted a harmonious style of living with nature and no money and everyone assigned a god-neighbor they could turn to. The opening to the modern world brought in money and devastated old social structures. Traditional housing broke down, destroying patterns of multi-generational living. Mun + Bogosian showed an ancient Chinese building, circular and built around a shared courtyard that housed around a hundred people. They quoted Jeremy Rifkin’s assertion that a sharing economy Is the future. Stressing working with the nonstandard body, they noted the husband and wife partnership of Arakawa and Gins that looked at multiple sciences to develop Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 28 procedural architecture, reversible design. Mun + Bogosian cited with approval8 their Bioscleave House with deliberately uncomfortable floors, challenging the body and even causing people to fall. Their claim is that this will stimulate their immune systems. Lars Spuybroek has developed human-machine production protocols able to tailor each unit to be adjusted to the individual. Francois Roche speaks of human-machine prototypologies that adapt construction to the human emotional body. Monitoring a human’s hand while the person hears a narrative can then set parameters for a building. Mun + Bogosian then related these varied ideas to their own efforts to rethink co-housing while adopting a neurological perspective. Michael Arbib introduced his discussion of “Building Community” by quoting Kevin Roche9: “The most important thing one can achieve in any building is to get people to communicate with each other. That’s really essential to our lives. We are not just individuals—we are part of a community. The old-time villages did that, and then we destroyed all that in the 19th century, when we started to build these vast expansions where there was no center, there is no community.” He then turned to the design by the Singapore architecture firm WOHA of Kampung Admiralty which won the 2018 World Architecture Festival Building of the Year. This is a "Vertical Kampung (village)", with a People's Plaza in the lower stratum, a Medical Centre in the mid stratum, and a Community Park with studio apartments for seniors in the upper stratum. The People’s Plaza is a fully public, porous and pedestrianized ground plane – i.e., it is not restricted to residents of the Kampung, but encourages interactions in and out of the complex. In the Plaza, the public can participate in organized events, join in the season’s festivities, shop, or eat at the hawker center on the 2nd story. Residents can actively come together to exercise, chat or tend community farms at the Community Park, an intimately-scaled, elevated village green. In addition, “buddy benches” at shared entrances encourage seniors to come out of their homes and interact with their neighbors. All this is in stark contrast to a “standard” high-rise apartment building in which people interact (if at all) only when riding the elevator to their isolated apartments. Relating this back to Baukultur, Arbib asked whether such a development could be the seed for a new approach to the mantra of Jane Jacobs, combining vertical villages (community on the small scale) with public parks, metro systems and the preservation of cultural sites of diverse importance (community on the large scale)? He closed by noting that WOHA’s breakthrough from designing individual houses came with winning two simultaneous competitions for two new metro stations in Singapore – thus in some sense anticipating the current issue of how to integrate far-flung living complexes with the historical and cultural sites of a large city. 8. Biology, Light, and Aesthetics Core Neuroscience: Circadian rhythms In addition to the visual mechanisms of the brain that support visual perception of the world around us(see below), the brain has mechanisms sensitive to light that help adjust our circadian rhythms. Satchin Panda studies the diverse circadian rhythms that relate to sleep, nutrition,

8 I’m not convinced. Read about the house’s discomforts here: https://www.nytimes.com/2008/04/03/garden/03destiny.html 9 As interviewed in http://www.archnewsnow.com/features/Feature512.htm Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 29 hormones, and how these related rhythms are disrupted by electrical light, with a host of possible adverse effects on the downtime that diverse bodily systems need, and that sleep provides. He found that many effects of light related to circadian rhythms persist in mice even in the absence of the rods and cones that provide the retinal receptors implicated in visual perception. He discovered melanopsin – a novel photo pigment in just a few thousand cells in the mammalian retina, hardwired to the master clock in the brain, more sensitive to blue light than other parts of the spectrum and thus less sensitive to the candlelight and firelight that our ancestors relied on at night. This clearly has implications both for lifestyle and for the way architects employ artificial light, especially during night time. Studies show that intermittent versus continuous lighting may be similar in effect on circadian rhythms, but one needs to start dimming lights 2 to 3 hours before bedtime. Light and Architecture Fred Marks discussed how light plays a crucial part in our lives and how architecture makes use of this. This includes celebratory purposes and seasonal timing, as at Stonehenge. Light is projected into the Pantheon in Rome via the oculus. Light is a symbol of knowledge and understanding and, yes, enlightenment. The Phoenix Public Library (Will Bruder) celebrates the summer solstice every June. Medieval castles were built with thick masonry walls that, for defensive purposes, contained only small apertures. Changes in European building practice from the Norman to the Gothic, such as flying buttresses, allowed walls to be thinner and allowed churches to include large stained glass windows. Glass windows in the renaissance brought more light through the walls, cast iron provided new ways of realizing this in the 19th century and, even more dramatically, steel girders in the 20th century removed weight support from the walls, which could then contain more glass and admit more light. One may then add a scrim to reduce the light, but how much is needed? Artificial light progressed from candles to gas, then electricity and now LEDs. Now more people stay awake late into the night, and 20% are on night shifts. Now light can be distributed to create engaging displays. Marks cited the Olafur Eliason installation, the Weather Project, at the Tate Modern in London in 2003. Representations of the sun dominated the expanse of the Turbine Hall and engaged many people, exemplifying the socialization induced by light. Many traditional farmers are out in the fields all day with bright light exposure except when going into the house for a nap or a meal. Panda now offers an App to help people monitor their light exposure, and finds that the built environment offers very different amounts of light. He measured his exposure to bright light in different venues and found that his office at the Salk offered surprisingly little exposure. Bright light that works well in the day is counter-productive experienced in, e.g., a 24 hour supermarket at night. With Marks, he analyzes the different light exposure of school children in different settings in San Diego. What about controlling warm versus blue lighting in the classroom? Well, a typical room has 4 sources of light, natural, overhead, task, and screens, so it is challenging to assess their relative contributions to the overall effect. Core Neuroscience: The Visual System The mammalian brain has two main pathways that link vision with perception (and much more). One travels from the retina to the superior colliculus in the midbrain and plays a key role in rapid involuntary eye movements, the other is the thalamocortical system that travel from Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 30 retina via thalamus to cerebral cortex. Tom Albright offered a quick tour of the thalamocortical system. Information reaching primary visual cortex then passes to diverse interacting visual areas of cerebral cortex including those involved in depth and motion processing and color, and then onward to object recognition – an assemblage of specialized processes collectively yielding visual experience of the world. David Hubel and Thorsten Wiesel showed that there are cells in primary visual cortex that act as “edge detectors,” firing most actively when an edge or gradient of a particular orientation appears at a particular location in the visual field. Albright offered a two-part theory concerning the early stages of visual processing: 1. The visual environment has measurable statistics. For example, angular correlation between contours decreases with the distance between them. Color correlation also declines, but in a different way. The statistics are different for a natural scene and for, e.g., a Jackson Pollock picture. 2. Organizational properties. Nearby features are represented nearby in the brain. Hubel and Wiesel’s edge detectors are arranged in “pinwheels” – the preferred orientation of cells changes progressively around the pinwheel, while nearby pinwheels link to nearby locales in the visual field. Anatomical connections link similar features. [Caveat: While nearby features are represented nearby in the brain, this breaks down in later regions.] Later regions of the visual system resolve the visual input by interpreting parts of the input as objects and recognizing the relations between them or the actions that are taking place. The bottom-up processing of the retinal input is complemented by top-down influences (often unconscious ones) based on hypotheses and expectations shaped by prior experience. In short, memory has a great impact on bottom-up processing -- a continuum from pure stimulus to pure imagery (and even hallucination). To illustrate this, Albright showed an image that most of the audience could not interpret. He then “changed their brains forever, without their permission” by showing the image of a bearded man from which the first image had been abstracted. Thereafter, one could not look at the first image without seeing the man’s face – and I can attest that this was the case for me even a year after first experiencing this effect. Thanks to memory, a visual environment can stimulate other senses as well as vision. What makes beauty? Tom Albright notes that the visual system evolved to detect relevant patterns in the environment, and continued his discussion by asking “What makes beauty.” His first answer is that the Fay Jones chapel in the woods and the Notre-Dame rose window and cable-stay bridges are all seen as beautiful because of regular changes in angle that relate to the properties of the low-level visual system. [However, a not-so-low-level system is crucial in that this beauty is based on relations between contours, not local Hubel-Wiesel features. How does one get from local edges to perceived contours in the visual scene? Computational models suggest that nearby neurons related to features consistent with a continuous contour tend to excite each other, while those that do not tend to inhibit each other (via inhibitory neurons), thus maintaining the activity that defines a possible contour consistent with the retinal stimulation.] To build on the above examples, Albright cited E.H. Gombrich on “the sense of order.” Gombrich contrasts Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 31

 Easy adjustment (familiarity) in perceiving a visual scene. Examples include regular tessellations and mandalas. Albright sees the Sydney Opera House as having conventional beauty based on easy adjustment.  Easy arousal (the sense of novelty). For this, he stresses the impact of memory on the bottom-up processing. A continuum from pure stimulus to pure imagery. The pattern requires assessing different hypotheses to impose sense on what you see. It requires “perceptual improvisation.” Easy arousal for Albright is the type of beauty one sees in non-representational art. He relates this to the notion of William James of “the victorious assimilation of the new” but does acknowledge Robert Irwin’s contrary view that such perception is “just Rohrschaching it.” I note, however, that our perception of any visual scene could be assessed in terms of easy adjustment versus easy arousal. Such a classification holds whether the scene is beautiful, ugly or just blah. Thus the dichotomy may be part of a theory of beauty, but does not answer the motivating question “What makes beauty?” It is also interesting to note that Albright’s account is based on “disembodied vision,” whereas Mallgrave approached Einfühlung in terms of embodied cognition, empathy and mirror neurons. Developing a new account that benefits from both these approaches poses a fine challenge for NfA. Albright is conducting with Sergei Gepshtein and a Baltimore architect a study of kids responding to patterns in a classroom, finding those which can reduce stress, or improve academic performance. He gave a talk on this at AIA, suggesting how NfA may improve the built environment for educating children, improving health, etc. Alison Whitelaw commented that Albright told us about vision, novelty ,and familiarity and then asked, “How do we trade these off in design?” E.O. Wilson speculates on the comfortable level of complexity. What is the impact of environments that support easy arousal and easy exploration? Consider increasing architectural complexity in the approach to an art gallery, but reducing it in the display galleries to provide a foil to support attention to the complexity of the artworks. Whitelaw has applied similar ideas to school design, to a public utility building (with public inclusion), and to a high school performing art center. Marcos Novak took us on a wild ride in his talk “Enacting civilization. Neuroaesthetics and thermodynamics of beauty,” starting on a musical note by playing Roxy Music’s Every dream home a heartache: “Standard of living is rising daily. ... penthouse perfection, but what goes on, what to do there? Better to pray there ... bungalow ranch style, all of its comforts seem so essential ... Inflatable doll, my breath is inside you ...”10 He asked how we can make things to “enact beauty.” Why do some civilizations build beautiful cities and others do not? Novak briefly explored diverse scientific theories. Let me note a number of his observations and ideas. He builds on notions of thermodynamics and Boltzmann’s explanation via statistical mechanics. A diagram of clock hand, all rounded at the center, narrowing to a point (the narrow PhD thesis) symbolized that everybody’s good at something but no one is minding the shop (e.g.,

10 Check out the lyrics! https://www.youtube.com/watch?v=nqXSAdOfCUE Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 32 global warming and catastrophe). But the sea urchin has many spikes, and each spike is not an end in itself, but one of many that contribute to the animal’s living. For an exhibition of his work in an Eisenman museum, he was faced with a 65 foot blank wall, but had no time to fill it with things. Overnight, he generated a program to define shapes in a limited supply of vinyl and specify how to cut them from the sheet and how to position each piece on the wall. What about an algorithm for placement in a a3D space, an algorithm where people can react in a shared virtual environment? World making, world shaping, relating building to poesis. His transLAB seeks to make things from which new species emerge. What then is a species if not defined by Darwin’s reproductive fitness? Mediated worlds. Beyond STEM and STEAM to incorporate the humanities and making. An array that’s more than n-dimensional in which one can locate efforts. Novak listed a breathless array of topics, from quantum mechanics to poetry to 20th century citizenship! On to beauty, which is above wisdom. Beauty is rare and it is difficult to achieve. But there are places where a community could create beauty together, as in Kyoto. Why did Athens spend half its military budget to build the Parthenon, a building that could last 2500 years? Maybe they were building for the future. So how can we build a beautiful civilization and save the world before we destroy it? The core meaning of aesthetics. is just “sensation.” It is the door, not the grand hall! John Zeisel. Warren Neidich. John Onians and his neuroarthistory. Neurocognitive poetics. The Evil Robots of the Ancient World – see the book Gods and Robots by Adrienne Mayor. We are deeply invested in expanding the scope of the human; expanding the degrees of freedom. Artificial gene synthesis, adding letters to the bases AGTC. Thermodynamics. Fourier analysis. Signal analysis. Building the city. Maximizing rent income is not the right constraint. If everybody has freedom, will the result be harmonic or beautiful. Why does time move in one direction? Why is entropy increasing? We are all unique. But unique does not imply equiprobable. Beauty is rare. Beauty is not increasing freedom, it must balance the freedom with constraints. A ladder from the superficial to the just, and the just is difficult. Life and consciousness are involved with the transcendent but also the material world. Novak’s final image was of Brancusi’s studio. Leach questioned the claim that beauty is rare. We can see a beautiful sunset almost every day. Novak responded that few planets have beautiful sunsets. Tourists flock to beautiful places and destroy them in the process. But they could go home and ask architects to design beautiful buildings for them. Leach responded that gay men can talk about beauty but perhaps others are too manly to talk about it!! Q: On her first day visit to Iceland, she was struck by the beauty of the sunset. On her second visit, in summer and with flocks of tourists, she did not find the sunset beautiful. Can neuroscience explain this? Novak responded that perhaps the first had rarity value that the second did not. [But this cannot be quite right. We can see sunset after sunset as beautiful. Perhaps the point is that if sunsets contain enough variety of shape and color, they invoke the easy arousal of which Albright spoke.] Gepshtein added that it’s early days for neuroaesthetics. To move forward, we need to understand the dynamic processes of appreciation in relation the impact of the new. [Recall the TV series of Bob Hughes on modern art, “The Shock of the New,” and the mention of William James by Albright.] Novak closed the discussion by stressing that Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 33 novelty does not guarantee beauty – car manufacturers come up with new exterior designs to sell more cars, not in the quest for beauty. 9. Neuromorphic and Dynamic Architecture Neuromorphic Architecture Michael Arbib introduced his theme of neuromorphic architecture in the sense of buildings (not just rooms) that in some sense have “brains.” A building then integrates a “body” (a possibly dynamic physical space) with a “neural” space that controls its interactions. We may view such a building as an inside-out robot or an embrained body. Form and space of the building are interactive, intertwined with actions and events in the life of its users. A building becomes a web of systems and components planned and constructed to address certain functionalities and yet which may combine to convey impressions, feelings, and aesthetic qualities. When we study animals and their behavior, we see that brain and body (including sensors and effectors) evolve together. This led him to the proposal for neuromorphic architecture that “Careful attention to “neural space” prior to commitment to the final form of the physical space may in future yield innovative designs that can then enrich the design of the building through constraint satisfaction between its physical and neural dimensions.” Arbib revisited Corbusier's dictum “A house is a machine for living in” and asked what this becomes in an age of cybernetic machines, rather than the ocean liners, automobiles and airplanes that inspired Corbusier. He noted that in Vers une architecture, Corbusier has two voices – one the admirer of the engineering aesthetic, the other of the purely non-functional architectural aesthetic of the Parthenon. We seek a cybernetic architecture that indeed captures the architect’s aesthetic as well as the cybernetician’s: “Machines” for efficiently supporting X, Y and Z while offering aesthetic pleasures. Will they/should they preserve Baukultur or radically transform it? He cited two challenge for Neuromorphic Architecture: homeostasis (regulating physiological parameters) as in Jean Nouvel's Institut du Monde Arabe in Paris, and supporting social interaction between a building and its users. Homeostasis is not “socially” interactive – the body adjusts to the ambient conditions or moves elsewhere (the latter option being less relevant for buildings). For social interaction we turn for inspiration to social cognitive neuroscience. Of particular relevance are mirror neurons which are involved both in our own actions and also in recognizing people acting in similar ways -- recognizing how the actions of others relate to our own repertoire, we can learn more about the world and prepare for the way we will interact in different social situations. However, the actions of a building will usually differ from the actions of humans (save when the building has part of a humanoid robot as a subsystem) and so “action recognition” rather than the specific mechanism of mirror neurons seems most relevant here – not merely recognizing the action of others but using that recognition to help decide what one should do next. The quest, then, is for brain operating principles that support a neuromorphic architecture for the “social interaction” of rooms or buildings with people in or near them, adapting buildings to the needs of their inhabitants. A classic example is provided by the Interactive Space Ada described below. Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 34

Arbib summarized the design of a reactive and adaptive intelligent kitchen put together by students in a course at USC.11 The kitchen was designed to help a person cook food based on a recipe. All computations were conditional on which recipe was selected, with the room keeping track of progress and helping as needed. The neural space included speech recognition, emotion recognition, event localization, and processing facial expressions. It also includes recipe management. Simulated “mirror neurons” came into the mix to track the manual actions of the cook. These had to be related to systems “beyond the mirror” to keep track of the state of preparation according to a particular recipe. A skillful chef would not want a kitchen like this, and even a novice cook could become frustrated by a kitchen that kept offering suggestions for cooking routines that the cook had by now mastered. This motivates bringing in emotion and learning as crucial themes for neuromorphic architecture. Someone who cooks rarely may need a lot of assistance, and even an experienced cook may find it helpful to receive timely reminders when working with the multiple components of a complex recipe. The room would use facial and vocal cues to recognize the emotional state of the user. Do they indicate frustration with excessive instruction or a measure of desperation in preparing the current dish? The room should be able to react accordingly. This leads into learning –both about what the user can handle without assistance and where help is needed (and this changes over time), and how the user chooses to modify the recipe so that the room can keep track of the user’s preferences. How will future buildings learn? In Stewart Brand’s (1995) book How buildings learn: What happens after they're built the focus is on how humans can reconfigure a building when the users have new needs or the neighborhood changes. Here, we ask how an embrained building with suitable effectors to reconfigure walls and other structures might itself change the building. Building Dynamics: Exploring Architecture of Change (Kolarevic & Parlac, 2015) offers ideas on buildings that can change shape but within a given range of functionality, in many but not all cases addressed more to entertainment and spectacle. To go beyond this, the memory of the building might extend building information modeling (BIM) not only to track changes in the building but also to plan and direct them This perspective might fit in with Macagno’s notion of lifespan architecture, where observing the changing capabilities of the inhabitants could factor into the building’s decisions. Ellie Al-Chaer introduced his work with architect Soulaf Abouras, extending the notion of responsive architecture – a term coined by Nicholas Negroponte -- through neuroscience-based material programming. He recalled Peter Cook’s fanciful walking city, constantly evolving, with diverse plug-in modules, perhaps an anticipation of the ideas of the previous paragraph. Soulaf developed a design for an architecture school where rooms can move to create different spaces. In the Al Bouchard towers, Abu Dhabi, the exterior (as in Nouvel’s Institut du Monde Arabe) was inspired by the mashrabiya. Its curtain wall has a lattice of units that open and close, controlled by sensorimotor mechanisms. Additionally, this adaptive curtain wall can rotate around the building.

11 For more on this and two other conceptual designs, see “Brains, machines and buildings: towards a neuromorphic architecture” (Arbib, 2012a). Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 35

Hygroskin covered a weather-sensitive pavilion of 2013. The changing surface can sense and react to humidity over the range of 30 to 90%. Here, the material itself is responsive rather than controlled by electric-powered motors. Diverse studies explore the use of innovative materials, inspired for example by the way Lycra can be stretched from a saggy piece of cloth to a shaped surface. The work even involves osteorobotics: Robot arms manipulate and stretch polycaptolactone (PCL) to build a structure -- and PCL is biodegradable. Design evolves from the material itself; making it self-emergent. Core Neuroscience: Neuroscience of Emotion Charles Darwin’s (1872) concern with relating facial expressions of certain animals to human emotional expression placed the external aspect of emotion within a comparative, and thus implicitly evolutionary, framework – in turn setting the stage for the notion that the internal role of emotions in affecting behavior has an evolutionary base with a strong social component. It is much debated whether there are basic emotions from which all others are constructed by some admixture that also involves cognition – Schadenfreude is certainly not one of them. In any case, an influential claim has been made by Paul Ekman (1999) that the basic emotions are anger, disgust, fear, happiness, sadness, and surprise. Fellous and Arbib (2005) edited a book that offered an integrative approach to both the neurobiology of emotion and attempts to endow robots and AI with at least simulacra of emotion that could aid human-machine interaction. In their chapter, Ortony, Norman and Revelle analyzed the interplay of affect (value), motivation (action tendencies), cognition (meaning), and behavior at three levels of information processing:  Reactive: a hard-wired releaser of fixed action patterns and an interrupt generator. This level has only proto-affect.  Routine: the locus of unconscious well-learned automatized activity and primitive and unconscious emotions.  Reflective: the home of higher-order cognitive functions, including metacognition, consciousness, self-reflection, and full-fledged emotions. They addressed the design of emotions in computational agents (these include “softbots” as well as embodied robots) that must perform unanticipated tasks in unpredictable environments. They argue that such agents, if they are to function effectively, must be endowed with curiosity and expectations and a sense of self that reflects parameter settings that govern the agent’s functioning. We will see examples of “artificial” emotion below. Here a very brief glimpse of some neurobiology: We cannot understand what brains contribute to our humanity unless we understand how emotion interacts with basic behaviors all the way up to higher cognition. Moreover, each part of the brain is a context for other brain regions, and they evolve in conversation with each other as they help contribute to survival. As we saw in Section 3 on measuring the brain, those who want to probe more deeply people’s emotional reactions (and other factors) to the built environment must include a detailed study of the autonomic nervous system. However, here we focus on the amygdala is a crucial part of the emotion system of the brain. It has been linked to fear behavior in mammals (LeDoux, 2000) and has been shown in humans to be related to the recognition of facial expressions that tell one that someone is to be feared or distrusted (Adolphs, Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 36

Tranel, & Damasio, 1998). People with lesions to the amygdala can still recognize other people on seeing their faces but can no longer make sound judgments as to the likelihood that a stranger should be mistrusted. Thus, our social interactions with others involve gaining a reliable impression of their general character (a reliable view of their “self”) as well as getting a feel for their current emotions and intentions. The amygdala is embedded in larger circuits, in particular those involving medial prefrontal cortex which link its immediate evaluations to the longer-term planning of behavior. The amygdala can influence cortical areas by way of feedback from proprioceptive or visceral signals or hormones and via projections to various "arousal" networks. It also has widespread influences on cognition and behavior through its cortical connections, while cognitive functions organized in prefrontal regions can in turn regulate the amygdala and its fear reactions. Emotions in a Neuromorphic Architecture Michael Arbib described the “emotions” of the Interactive Space Ada developed for the Swiss Expo of 2002 by a team led by the computational neuroscientists Rodney Douglas and Paul Verschure (Eng et al., 2003). Ada had a “brain” based (in part) on artificial neural networks. She had “emotions” and “wanted” to play with her visitors. She continually evaluated the results of her actions and expressed emotional states accordingly, and tried to regulate the distribution and flow of visitors accordingly. When people participate, Ada is happy; when they do not, she is frustrated. Her level of overall happiness is translated into the soundscape and the visual environment in which the visitor is immersed, establishing a closed loop between environment and visitor. JG Ballard’s 1962 short story The Thousand Dreams of Stellavista (recommended to me by Branko Kolarevic) explored the perils of “psychotropic” buildings that learn the emotional patterns of their inhabitants and express them through dynamic changes of color and form whose persistence may prove catastrophic for the next inhabitant. Ehnaz Farahi described her cutting-edge research showing how computation can be embedded in materials, affecting tactility, texture and appearance to yield emotive matter. Nature is a source of amazing forms and patterns. What can design learn from these? Farahi stresses human-material interaction over human-machine interaction. Inspiration can come in diverse forms. One can even learn from how trees respond to their environment to define a new symbiosis. She is influenced the work of Rosalind Picard12 at MIT on affective computing and ways of following people’s emotion, but her focus is the design of soft, material reconformable materials that can establish empathic relations with humans. Dramatic videos showed devices around the human body that move in relation to EEG, opening and closing a helmet and varying the boundary between the body and the environment – something Farahi relates to the notion of extended mind as studied by Andy Clark and David Chalmers. Fish scales – hard elements on a semi flexible mesh system -- provide inspiration for new materials -

12 http://web.media.mit.edu/~picard/ Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 37

One system maps the sensed direction of the onlooker’s gaze into movements of one’s garment. H, providing haptic and visual response to the others gaze. How might this affect one’s understanding of social relations? Farahi builds on Ekman’s 6 basic emotions to incorporate a system that can recognize emotions from their facial expression. She then shows how these can be mapped onto clothing. What if one’s garments provide a second skin extending our range of emotional expression? Perhaps such garments would help those with autism better understand the emotions of others and project their own. Exploiting the power of affective computing, Farahi designed an interactive display case for Adidas, creating positive emotions for a new product. The shoe in the box becomes “alive” and tracks the viewer -- this gets interesting when there are multiple viewers. Another piece was inspired by hummingbird iridescence changing color to attract a mate. This required selecting the materials and determining how they are moved. The extra challenge was that the system had to be robust enough to last for a one year display in Chicago. How might sensory experience affect social interaction? The overall challenge is to establish an empathic relation between the body and the environment. Asked how this might relate to interior design, Farahi answered that she started as architect but has come to focus on the body. However, her thesis presents a bio-inspired methodology, linked to emotion, that can be applied at many different scales. Q. Sees future of increased communication with the environment. Not just voice assistants. Buildings that embrace us. That help us to heal. Embracing nature in a new way. A building that responds if I am scared or feel alone. Providing a supportive atmosphere. Buildings as robots Guvenc Ozel works with spatial intelligence, cyberphysical architecture, and brain-computer interfaces, integrating data with architectural context. In his lab at UCLA, he doesn’t construct buildings but looks at buildings as robots. Robots can construct in physical and virtual worlds, maybe yielding integrated forms. He sees the key for the 4th industrial revolution not as automation but as a freeing of cybernetic systems from human control: intelligence for non- organic actors in a social context; active participants in human social interaction. The result will be an architecture that relies on media and virtual depictions as much as physical constructions, integrating diverse cyberphysical technologies. One example – a space that tracks your actions and will in some cases react in real time, a form of spatial intelligence. Note the importance of predictions of future possibilities to determine interactions accordingly with humans. A video showed a machine learning system tracking and identifying objects and agents in a street scene video. Meta-parametricism (recall Leach on GANs) yields novel human faces - borderline creativity. Augmented reality relying on machine learning and diverse sensors adds to the external world. 7 years ago: An agent changing form based on EEG, rather than responding to physical actions of the human body. Venice Architecture Biennale 5 years ago: An augmented reality project, expanding architecture. What can be embedded in physical environments? Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 38

Coachella music festival proposal: Robotically actuated carbon fibers to changes the shape of a structure, with visually projected overlay. Future project? A robot construction system that could gather materials on Mars and make decisions based on the gathered materials. Ozel also looks to machine learning for interdisciplinary design in a social setting, as in generating individually tailored dynamic facades for buildings. [Recall the design for this in Minority Report – an example of world building by Alex McDowell with whom Sergei Gepshtein collaborates in USC’s 5D|World Building Institute.] One can create inherently virtual environments that are fictitious rather than mimicking natural environments. These become architectural expressions in their own right. And one can study turning virtual systems into architectural representations. Think of architectural objects as robots. Make them generative - a design tool to help humans work collaboratively. Ozel looks to machines as creative collaborators. There are spectra of understanding other than the human and there are other examples from nature, but we can go beyond models from nature. About the Author: Michael Arbib is a pioneer in the study of computational models of brain mechanisms, especially those linking vision and action, and their application to artificial intelligence and robotics. Currently his two main projects are “how the brain got language” through biological and cultural evolution as inferred from data from comparative (neuro)primatology, and the conversation between neuroscience and architecture. He serves as Coordinator of ANFA’s Advisory Council and is currently Adjunct Professor of Psychology at the University of California at San Diego and a Contributing Faculty Member in Architecture at NewSchool of Architecture and Design. The author or editor of more than 40 books, Arbib is currently at work on When Brains Meet Buildings, integrating exposition of relevant neuroscience with discussions of the experience of architecture, the design of architecture, and neuromorphic architecture. References Adolphs, R., Tranel, D., & Damasio, A. R. (1998). The human amygdala in social judgment. Nature, 393(6684), 470-474. Albus, J. S. (1971). A theory of cerebellar function. Math. Biosci., 10, 25-61. Arbib, M. A. (1972). The Metaphorical Brain: An Introduction to Cybernetics as Artificial Intelligence and Brain Theory. New York: Wiley-Interscience. Arbib, M. A. (2012a). Brains, machines and buildings: towards a neuromorphic architecture. Intelligent Buildings International, 4(3), 4(3), 147-168, DOI:110.1080/17508975.17502012.17702863. Arbib, M. A. (2012b). How the Brain Got Language: The Mirror System Hypothesis. New York & Oxford: Oxford University Press. Arbib, M. A. (2016). Towards a Computational Comparative Neuroprimatology: Framing the Language-Ready Brain. Physics of Life Reviews, 16, 1-54. Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 39

Arbib, M. A. (2017). How Language Evolution Reshaped Human Consciousness. In R. R. Poznanski, J. A. Tuszynski, & T. E. Feinberg (Eds.), Biophysics of Consciousness: A Foundational Approach (pp. 87-128). Singapore: World Scientific. Arbib, M. A. (2019a). The aboutness of language and the evolution of the construction-ready brain. In A. Lock, C. Sinha, & N. .Gonthier (Eds.), The Oxford Handbook of Symbolic Evolution (pp. In press). Oxford: Oxford University Press. Arbib, M. A. (2019b). Poetics and More in Performative Architecture: Towards a Neuroscience of Dynamic Experience and Design. In K. Mitra (Ed.), The Routledge Companion to Paradigms of Performativity in Design and Architecture: Using Time to Craft an Enduring, Resilient and Relevant Architecture: Routledge. Arbib, M. A. (Ed.) (1995). The Handbook of Brain Theory and Neural Networks Cambridge, MA: A Bradford Book/The MIT Press. Arbib, M. A., Aboitiz, F., Burkart, J., Corballis, M., Coudé, G., Hecht, E., . . . Wilson, B. (2018). The Comparative Neuroprimatology 2018 (CNP-2018) Road Map for Research on How the Brain Got Language. Interaction Studies, 19(1-2), 370-387. Arbib, M. A., & Hesse, M. B. (1986). The Construction of Reality. Cambridge: Cambridge University Press. Arbib, M. A., & House, D. H. (1987). Depth and Detours: An Essay on Visually-Guided Behavior. In M.A.Arbib & A.R.Hanson (Eds.), Vision, Brain, and Cooperative Computation (pp. 129- 163). Cambridge, MA: A Bradford Book/MIT Press. Arnheim, R. (1977). The Dynamics of Architectural Form. Berkeley, Los Angeles: University of California Press. Bliss, T. V., & Gardner-Medwin, A. R. (1973). Long-lasting potentiation of synaptic transmission in the dentate area of the unanaestetized rabbit following stimulation of the perforant path. J Physiol, 232(2), 357-374. Boden, M. A. (2006). Mind as Machine: A History of Cognitive Science. New York: Oxford University Press. Brand, S. (1995). How buildings learn: What happens after they're built: Penguin. Buccino, G., Lui, F., Canessa, N., Patteri, I., Lagravinese, G., Benuzzi, F., . . . Rizzolatti, G. (2004). Neural circuits involved in the recognition of actions performed by nonconspecifics: an FMRI study. J Cogn Neurosci, 16(1), 114-126. Darwin, C. (1872). The expression of the emotions in man and animals (republished in 1965). Chicago: University of Chicago Press. Davos. (2018a). Towards a high-quality Baukultur for Europe. Davos, Switzerland: Conference of Ministers for Culture. Davos. (2018b). Towards a high-quality Baukultur for Europe: Context Document. Davos, Switzerland: Conference of Ministers for Culture. di Pellegrino, G., Fadiga, L., Fogassi, L., Gallese, V., & Rizzolatti, G. (1992). Understanding motor events: a neurophysiological study. Exp Brain Res, 91(1), 176-180. Eberhard, J. P. (2008). Brain Landscape: The Coexistence of Neuroscience and Architecture. Oxford, New York: Oxford University Press. Ekman, P. (1999). Basic Emotions. In T. Dalgleish & M. Power (Eds.), Handbook of Cognition and Emotion (pp. 45-60). New York: John Wiley & Sons. Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 40

Ellard, C. (2018). The psychogeography of urban form: Building and measuring Baukultur. Intertwining: Unfolding art and science, 01, 92-101. Eng, K., Klein, D., Babler, A., Bernardet, U., Blanchard, M., Costa, M., . . . Manzolli, J. (2003). Design for a brain revisited: the neuromorphic design and functionality of the interactive space'Ada'. Reviews in the Neurosciences, 14(1-2), 145-180. Fellous, J.-M., & Arbib, M. A. (Eds.). (2005). Who Needs Emotions: The Brain Meets the Robot. Oxford, New York: Oxford University Press. Gallese, V., Fadiga, L., Fogassi, L., & Rizzolatti, G. (1996). Action recognition in the premotor cortex. Brain, 119, 593-609. Gibson, J. J. (1966). The Senses Considered as Perceptual Systems. Boston: Houghton Mifflin. Gregory, R. L. (1967). Will Seeing Machines have Illusions? In N.L.Collins & D.Michie (Eds.), Machine Intelligence 1: Oliver & Boyd. Guazzelli, A., Corbacho, F. J., Bota, M., & Arbib, M. A. (1998). Affordances, Motivation, and the World Graph Theory. Adaptive Behavior, 6, 435-471. Hebb, D. O. (1949). The Organization of Behavior. New York: John Wiley & Sons. Inoue, M. (1985). Space in Japanese Architecture (H. Watanabe, trans.). New York & Tokyo: Weatherhill. (Original work published 1969). Iriki, A., & Taoka, M. (2012). Triadic (ecological, neural, cognitive) niche construction: a scenario of human brain evolution extrapolating tool use and language from the control of reaching actions. Philosophical Transactions of the Royal Society B: Biological Sciences, 367, 10-23 Jelić, A., Tieri, G., De Matteis, F., Babiloni, F., & Vecchiato, G. (2016). The Enactive Approach to Architectural Experience: A Neurophysiological Perspective on Embodiment, Motivation, and Affordances. Frontiers in Psychology, 7, https://www.frontiersin.org/article/10.3389/fpsyg.2016.00481. doi:10.3389/fpsyg.2016.00481 Julesz, B. (1971). Foundation of Cyclopean Perception. Chicago: University of Chicago Press. Kaas, J. (Ed.) (2017). Evolution of Nervous Systems (Second Edition; in 4 volumes): Elsevier. Kolarevic, B., & Parlac, V. (Eds.). (2015). Building Dynamics: Exploring Architecture of Change. London and New York: Routledge. Laland, K. N., Odling-Smee, J., & Feldman, M. W. (2000). Niche construction, biological evolution, and cultural change. Behavioral and Brain Sciences, 23(01), 131-146. doi:doi:10.1017/S0140525X00002417 LeDoux, J. E. (2000). Emotion circuits in the brain. Annu Rev Neurosci, 23, 155-184. Lynch, K. (1960). The image of the city. Cambridge, MA: MIT press. Marr, D. (1969). A theory of cerebellar function. J Physiol, 202(43770.46). Marr, D. (1982). Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. New York: W.H. Freeman. Neisser, U. (1976). Cognition and Reality: Principles and Implications of . : W.H. Freeman. O'Keefe, J., & Dostrovsky, J. O. (1971). The hippocampus as a spatial map: preliminary evidence from unit activity in the freely moving rat. Brain Research, 34, 171-175. O'Keefe, J., & Nadel, L. (1978). The Hippocampus as a Cognitive Map. Oxford: Oxford University Press. Michael Arbib: August in San Diego: Neuroscience for Architecture, Urbanism & Design 41

Pribram, K. H. (1960). A review of theory in physiological psychology. Annual Rev. Psychol., 11(1- 40). Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychol. Rev., 65, 386-408. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning Internal Representations by Error Propagation. In D. Rumelhart & J. McClelland (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol.1 (pp. 318-362). Cambridge, MA: The MIT Press. Samuel, A. L. (1959). Some Studies in Machine Learning Using the Game of Checkers. IBM. J. Res. and Dev., 3, 210-229. Sbriscia-Fioretti, B., Berchio, C., Freedberg, D., Gallese, V., & Umiltà, M. A. (2013). ERP Modulation during Observation of Abstract Paintings by Franz Kline. PLoS One, 8(10), e75241. doi:10.1371/journal.pone.0075241 Schacter, D. L., Addis, D. R., Hassabis, D., Martin, V. C., Spreng, R. N., & Szpunar, K. K. (2012). The future of memory: remembering, imagining, and the brain. Neuron, 76(4), 677-694. Scoville, W. B., & Milner, B. (1957). Loss of Recent Memory After Bilateral Hippocampal Lesions (Reprinted in J Neuropsychiatry Clin Neurosci 2000, 12, pp.103-113). J. Neurol. Neurosurg. Psychiatr., 20, 11-21. Siddharth, Patel, A. N., Jung, T.-P., & Sejnowski, T. J. (2018). A Wearable Multi-modal Bio-sensing System Towards Real-world Applications. IEEE Transactions on Biomedical Engineering, 66(4), 1137-1147. Sutton, R. S. (1988). Learning to predict by the methods of temporal differences. Machine Learning, 3, 9-44. Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: An Introduction. Cambridge, MA: The MIT Press. Ulrich, R. (1984). View through a window may influence recovery. Science, 224(4647), 224-225. Wicker, B., Keysers, C., Plailly, J., Royet, J. P., Gallese, V., & Rizzolatti, G. (2003). Both of us disgusted in My insula: the common neural basis of seeing and feeling disgust. Neuron, 40(3), 655-664. Zeidman, P., & Maguire, E. A. (2016). Anterior hippocampus: the anatomy of perception, imagination and episodic memory. Nat Rev Neurosci, 17(3), 173-182. doi:10.1038/nrn.2015.24 Zeisel, J. (2006). Inquiry by design: Environment/behavior/neuroscience in architecture, interiors, landscape, and planning: WW Norton & Co.