PROFILE

Profile of David Heeger PROFILE Brian Doctrow, Science Writer

“Interdisciplinary” has become a buzzword in science research project, a collaboration in recent years. According to David Heeger, a profes- between computer scientist sor of and neural science at New York and psychol- University (NYU), “You don’t do interdisciplinary sci- ogist Jacob Nachmias. ence by putting a physicist and a biologist in the same After graduating with a office. Nor do you make interdisciplinary science hap- bachelor’s degree in mathe- pen by taking a grant proposal and assigning it for matics in 1983, Heeger de- review to people from a different field who know nothing cided to continue studying about it. You have to be willing to learn the other field’s with Bajcsy, earning his master’s language...techniques...and literature.” Heeger, who degree in 1985 and his PhD in was elected to the National Academy of Sciences in 1987, both in computer sci- 2013, understands interdisciplinary science more than ence. As a thesis advisor, Bajcsy most. For decades, Heeger’s research has straddled the was generous, encouraging boundaries between psychology, , and Heeger to publish his disserta- computer science, with lasting impacts on all three fields. tion work as sole author and providing him with valuable David Heeger. Image courtesy of the Na- Early Opportunities professional contacts, including tional Academy of Sciences. David Heeger was born into science. His father, Alan Alexander Pentland, who coad- Heeger, is a physicist whose discovery of conductive vised Heeger on some of his dissertation work, and polymers earned him the Nobel Prize for Chemistry in Edward Adelson, who would later become Heeger’s 2000. In addition, his brother, Peter, is a transplant postdoctoral advisor at the renowned Massachusetts immunologist at the Icahn School of Medicine at Institute of Technology (MIT) Media Lab. For his dis- Mount Sinai in New York. Heeger believes that his sertation research, Heeger developed algorithms for father’s work at the interface between physics and extracting velocity from an image sequence or video, chemistry influenced his own interest in working inspired by the then-current understanding of how the across scientific disciplines. He was thrilled when his brain accomplishes the same task. One of the articles father won the Nobel Prize, recalling that “[my father] describing this research was awarded the first biennial called and woke me up at 6:00 that morning, and I David Marr Prize from the International Conference on woke up my family screaming!” in 1987 (1). As an undergraduate, Heeger attended the Uni- versity of Pennsylvania, mainly because his father was The Normalization Model on the faculty at the time, and tuition was free. It was As a postdoctoral fellow at MIT, Heeger continued his during his senior year at Penn that he was first exposed work on image processing. He also began working on to the field of vision science. At the time, there was a computational neuroscience—understanding brain thriving vision research group at Penn, which included function in terms of information processing. This work faculty from engineering, psychology, and the medi- would eventually lead to the development of Heeger’s cal school. Heeger would sit in on the group’s in- “normalization model” of neural computation. terdisciplinary seminar series, learning how these To illustrate how the normalization model works, seemingly disparate fields intersected in studying vision, imagine a neuron in the brain’s primary visual cortex how brain function could be understood in terms of in- that responds to images with a vertical edge but not to formation and signal processing, and how ideas from images with a horizontal edge. An image containing perceptual psychology could be used to develop com- both horizontal and vertical edges will produce a puter programs for image and video analysis. The in- weaker response in this neuron than one with a vertical terdisciplinary nature of vision research formed the edge alone, a phenomenon known as cross-orienta- basis of Heeger’s subsequent research interests. While tion suppression. According to the normalization still an undergraduate, Heeger participated in his first model, this happens because a second neuron sensitive

This is a Profile of a recently elected member of the National Academy of Sciences to accompany the member’s Inaugural Article on page 1773.

www.pnas.org/cgi/doi/10.1073/pnas.1616435114 PNAS | February 21, 2017 | vol. 114 | no. 8 | 1745–1747 Downloaded by guest on September 27, 2021 to horizontal edges responds at the same time, and the rivalry and measured the speed of those waves (4). output of each neuron is divided by the combined re- Heeger and his team have used fMRI to test theories sponse of both. This division, analogous to normalizing a of how the brain visually perceives color, motion, vector in mathematics, acts as a type of automatic gain pattern, and texture, and have contributed to research control, preventing the combined signal from getting on dyslexia and autism. too large and overloading the neural circuit. Heeger’s original paper on the normalization , Processing Timescales, and Cortical model (2), published in 1992, has been cited more Function than 1,400 times. In subsequent work, Heeger and his In 2002, Heeger left Stanford and moved to NYU. “I collaborators demonstrated quantitative agreement think it was a good time in my career to make a move,” between experimental data and the predictions of the Heeger says. NYU was home to a large interdisciplinary normalization model. Meanwhile, other researchers group of researchers in psychology, neuroscience, have found that the model can explain neuronal and computer science, some of whom Heeger had function in a wide variety of neural systems and spe- known and collaborated with since he was a gradu- cies, from decision-making in primates to olfaction in ate student. Heeger relished the opportunity to join fruit flies. Of the model, Heeger says, “I continue to be this group. While at NYU, Heeger incorporated surprised at the impact that it’s had, and how far this normalization into a computational model that de- one simple idea can go.” scribes how changes in visual attention modulate the responses of neurons in the visual cortex (5). This Stanford and fMRI model could reconcile a variety of seemingly con- In 1990, after finishing his postdoctoral fellowship, flicting empirical results. Heeger joined the vision research group at the Ames Heeger continues to explore new ways to study Research Center of the National Aeronautics and the brain. One of his recent projects involves using Space Administration (NASA) in California, where he Hollywood movies as stimuli to drive simultaneous had become acquainted with a group of researchers activity in multiple brain areas in a highly controlled during a vision science conference. At the time, this manner. This idea came from a postdoctoral asso- group focused on understanding aspects of vision that ciate in Heeger’s laboratory at the time, Uri Hasson, applied to aircraft pilots, as part of NASA’s aeronautics who showed that movies elicit similar activity patterns missions. Heeger spent more than a year at NASA in the brains of different people. Or as Heeger puts it, before taking a faculty job at nearby Stanford Uni- “Hollywood is better at taking control over your brain versity, where he would spend the next decade. than any cognitive psychologist has ever been.” By It was around the time Heeger moved to Stanford breaking films into segments of varying length and that the first articles were published on using magnetic scrambling the order of the segments, Hasson, resonance imaging (MRI) to measure brain activity. At Heeger, and their collaborators identified a hierarchy the same time, the Stanford Medical School opened of brain areas that process information at different an MRI research center. Heeger recalls that the di- timescales (6). rector of the new center e-mailed the Stanford faculty Heeger’s Inaugural Article (7) exemplifies the in- asking if there was anything the center could do for terdisciplinary nature of his research interests, in- them. Heeger’s colleague Brian Wandell replied, corporating aspects of signal processing, neuroscience, asking if they could conduct fMRI experiments. “The perceptual psychology, computer vision, and artificial next thing I know,” says Heeger, “I’m in Brian’s garage intelligence. In the article, Heeger proposes a new on a weekend, sawing a bunch of two-by-fours to computational model with far-reaching goals: to ex- build a projection screen, so that we can show visual plain how the brain integrates sensory input with prior images to people while they’re in the MRI scanner.” knowledge or expectations. As Heeger explains, “Ican For Heeger, fMRI research provided yet another op- instruct you to imagine Woody Allen, or Bill Clinton’s portunity to combine perceptual psychology and face, or what your living room looks like. And when you neuroscience with image processing and computer do that, there’s activity in the visual part of your brain vision, because image-processing algorithms had to that looks pretty much like you were actually looking at be developed to analyze the data produced by the the picture. This means that visual perception and visual fMRI measurements. Heeger’s first fMRI study (3), imagery are linked, accomplished by the same circuits published in 1996, has been cited nearly 2,000 times. and processing in the brain.” Furthermore, the re- Heeger emphasizes that his use of fMRI is different sponse of neurons in the visual cortex to an image is from most others’. Most studies focus always delayed by 50–200 milliseconds. “So how can on determining which parts of the brain are involved in you catch a baseball if the perceptual processing is a a particular task. However, the seat of visual process- quarter of a second delayed, and then it takes another ing in the brain is already known. Instead, Heeger quarter of a second to program the movement?” measures how the activity in previously identified Heeger asks. “You’ll reach out to catch it after it’sal- brain regions changes during different tasks and in ready passed by.” In Heeger’s model, as sensory in- response to different stimuli. For example, Heeger formation is processed through layers of neurons, each and his colleagues discovered waves of activity that layer provides feedback to the previous layer based on travel across the surface of the brain coincident with prior expectations. Thus, the brain can use the same the perception of a visual illusion called binocular type of computations to perceive external images and

1746 | www.pnas.org/cgi/doi/10.1073/pnas.1616435114 Doctrow Downloaded by guest on September 27, 2021 generate images from memory by varying the weight both neuroscience research and the design of given to prior knowledge. Incorporating prior knowl- machine learning algorithms with artificial neural edge also allows prediction. networks. Like the normalization model, Heeger de- Heeger sees the Inaugural Article as an attempt scribes the theory as a pretty simple idea that does to develop a unified theory of cortical function—an some interesting things. “We’ll see how far it goes,” empirically testable theoretical framework for guiding he says.

1 Heeger DJ (1988) Optical flow using spatiotemporal filters. Int J Comput Vis 1(4):279–302. 2 Heeger DJ (1992) Normalization of cell responses in cat striate cortex. Vis Neurosci 9(2):181–197. 3 Boynton GM, Engel SA, Glover GH, Heeger DJ (1996) Linear systems analysis of functional magnetic resonance imaging in human V1. J Neurosci 16(13):4207–4221. 4 Lee SH, Blake R, Heeger DJ (2005) Traveling waves of activity in primary visual cortex during binocular rivalry. Nat Neurosci 8(1):22–23. 5 Reynolds JH, Heeger DJ (2009) The normalization model of attention. Neuron 61(2):168–185. 6 Hasson U, Yang E, Vallines I, Heeger DJ, Rubin N (2008) A hierarchy of temporal receptive windows in human cortex. J Neurosci 28(10): 2539–2550. 7 Heeger DJ (2017) Theory of cortical function. Proc Natl Acad Sci USA 114:1773–1782.

Doctrow PNAS | February 21, 2017 | vol. 114 | no. 8 | 1747 Downloaded by guest on September 27, 2021