The Center for Brains, Minds and Machines Overview of the Center Second CBMM Summer School, 2015 The Center for Brains, Minds and Machines Vision Accumulated knowledge and technology, now in place, enables a rapid leap in our scientific understanding of intelligence and our ability to replicate intelligence in engineered systems. Mission We aim to create a new field by bringing together computer scientists, cognitive scientists and neuroscientists to work in close collaboration. The new field – the Science and Engineering of Intelligence – is dedicated to developing a computationally centered understanding of human intelligence and to establishing an engineering practice based on that understanding. Why now Machine Learning Computer Science Neuroscience Computational Cognitive Science Neuroscience Science+ Technology of Intelligence3 Centerness: collaborations across different disciplines and labs MIT Harvard Boyden, Desimone ,Kaelbling , Kanwisher, Blum, Kreiman, Mahadevan, Katz, Poggio, Sassanfar, Saxe, Nakayama, Sompolinsky, Schulz, Tenenbaum, Ullman, Wilson, Spelke, Valiant Rosasco, Winston Rockefeller Allen Institute UCLA Stanford Cornell Freiwald Koch Yuille Goodman Hirsh Hunter Wellesley Puerto Rico Howard Epstein,Sakas, Hildreth, Conway, Bykhovaskaia, Ordonez, Manaye, Chouikha, Chodorow Wiest Arce Nazario Rwebargira CBMM Organizational Chart Dean of the School of Science, MIT Robert Desimone, Director, McGovern Institute for Brain Research at MIT Education Evaluation Director External Advisory Committee UIUC: Lizanne DeStefano MIT: Tomaso Poggio Center Director Associate Director Associate Director Managing Director MIT: Kathleen HU: L Mahadevan MIT: Matt Wilson HU: Kenneth Blum Sullivan HU: Liz Spelke Education Coordinator KT Coordinator Research Coordinator Diversity Coordinator WC: Ellen Hildreth MIT: Boris Katz MIT: Patrick Winston MIT: Mandana Sassanfar CU: Haym Hirsh HU: L Mahadevan T3 Visual T1 Developing T2 Circuits for T4 Social T5 Theory for Intelligence Intelligence Intelligence Intelligence Intelligence MIT: Shimon MIT: Josh HU: Gabriel MIT: Nancy MIT: Tomaso Ullman Tenenbaum Kreiman Kanwisher Poggio MIT: Boris Katz The Center for Brains, Minds and Machines A little bit of history and background 6 Why now: recent progress in AI Why now: very recent progress in AI 10 11 Why now: very recent progress in AI 14 15 Thus we see great advances in AI based on machine learning research of 20 years ago … but we are still very far from understanding human intelligence and the brain 16 What is this? What is Hueihan doing? What does Hueihan think about Joel’s thoughts about her? Intelligence and Turing++ Questions • Intelligence —> Human Intelligence • (Human) Intelligence: one word, many problems • A CBMM mission: define and “answer” these Turing++ Questions 18 Turing++ Questions functi onal theor y CB M M th eo ry The challenge is to develop computational models that answer questions about images and videos such as what is there / who is there / what is the person doing and eventually more difficult questions such as who is doing what to whom? • what happens next? at the computational, psychophysical and neural levels. The who question: face recognition from experiments to theory (Workshop, Sept 4-5, 2015) Model ML AL AM Neural Circuits of Thrust 5 Intelligence Social Visual Intelligence Intelligence Thrust 1 More about CBMM Second CBMM Summer School, 2015 Industrial partners IIT A*star Hebrew U. MPI Metta, Rosasco, Shashua Buelthoff Sandini Tan Genoa U. Weizmann City U. HK NCBS Smale Verri Ullman Raghavan Google IBM Microsoft Siemens Schlumberger GE Norvig Lemnios Blake Boston DeepMind Orcam Rethink Robotics MobilEye Dynamics Shashua Shashua Hassabis Raibert Brooks Adding value in Reserach Thrust 1: Thrust 2: Development of Circuits for Intelligence Intelligence Josh Tenenbaum Gabriel Kreiman Thrust 5: Theory of Intelligence Tomaso Poggio Thrust 3: Thrust 4: Visual Intelligence Social Intelligence Shimon Ullman Nancy Kanwisher a. Thrust 1 The goal of Thrust 1 is to understand the roots of human intelligence o Infants’ early physical representations: Forces and masses o Learning physics from dynamic scenes o Unifying frameworks for intuitive physics beyond rigid, solid objects o Efficiency and experience in infants’ goal representations o Computing Efficiency in Infancy o Efficiency and effort: Bridging intuitive psychology and intuitive physics o Social cognition and the naïve utility calculus o Do children expect others to learn rationally from evidence? o Inference of mental states from emotional expressions o How do children evaluate potential social partners? o Early reasoning about affiliation and kinship o Children’s perception of causal relations across intuitive physics and psychology o Knowing what to look for, in object perception and exploratory learning o Building more sophisticated models of children’s learning in goal-directed search o Lookit: An online developmental lab o Probabilistic programs for scene understanding o Hierarchical motion perception b. Thrust 2 Thrust 2 seeks to discover the neural circuits that answer Turing++ Questions such as “What and who are there? What are they doing?” To achieve this goal, scientists in Thrust 2 record electrical signals from the brains of humans and experimental animals while they are identifying objects and interpreting scenes, and then build computational models of the circuit function. • Towards a biologically inspired human-level language processing system • Decoding human action and interaction in the human brain • Top-down processes in extracting meaning from images • Task-dependence in visual processing • Neural mechanisms of face detection • Neural circuits underlying navigation • Computational architectures mediating object recognition under occlusion • Biological and computational code underlying pattern completion • Light field Neural Imaging • Expansion Microscopy • Scalable Neural Recording Probes c. Thrust 3 Thrust 3 aims to determine the elements involved in fully understanding visual scenes. Our computational models should be able to extract from the scene information about actions, agents, goals, object configurations, social interactions, and more. • Answering questions about videos • Disambiguating language and video • Learning Vision and Language with weak supervision • Learning attention mechanisms and segmentation • Visual processing with minimal recognizable configurations • Human language learning • Understanding intentions and goals in human interactions from gaze direction • Towards a biologically inspired human-level language processing system • Dataset development d. Thrust 4 The goal of Thrust 4 is to understand the capabilities and brain-bases of various aspects of social perception. • Functional organization of social perception and cognition in the superior temporal sulcus • Face Looking Behavior and Image Statistics for Faces in the Real World • Fast reading of Action Goals • Decoding human action and interaction in the human brain • Understanding intentions and goals in human social interactions from gaze direction e. Thrust 5 The Theoretical Thrust aims to provide understanding, guide computer implementations, and inform and interpret experiments for the other thrusts. • Biologically Plausible Implementations of i-theory for face and object recognition • Invariance and Selectivity in Visual Cortex • Invariant Representation Learning for Speech Recognition • Invariant representations for action recognition in the human visual system • Learning and Reasoning in Symbolic Domains • Probabilistic Learning and Inference in a Neuroidal Model of Computation • The computational role in object recognition of eccentricity dependent resolution in retina • Computational architectures mediating object recognition with occlusion (with Thrust 2) • The development of geometrical intuitions • Visual routines • Compositional Models of Humans and Animals • Learning Vision and Language with weak supervision 28 A summary To understand (human) intelligence, we must: • Understand what we compute • How what we compute develops • How amplified by social interaction • How implemented in neural tissues.