DOCSLIB.ORG

An Algorithmic Perspective on Imitation Learning

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
An Algorithmic Perspective on Imitation Learning Foundations and Trends ® in Robotics Vol. 7, No. 1-2 (2018) 1–179 © 2018 T. Osa, J. Pajarinen, G. Neumann, J. A. Bagnell, P. Abbeel and J. Peters DOI: 10.1561/2300000053 An Algorithmic Perspective on Imitation Learning Takayuki Osa Joni Pajarinen University of Tokyo Technische Universität Darmstadt [email protected] [email protected] Gerhard Neumann J. Andrew Bagnell University of Lincoln Carnegie Mellon University [email protected] [email protected] Pieter Abbeel University of California, Berkeley [email protected] Jan Peters Technische Universität Darmstadt [email protected] Contents 1 Introduction 3 1.1 Key successes in Imitation Learning . 4 1.2 Imitation Learning from the Point of View of Robotics . 5 1.3 Differences between Imitation Learning and Supervised Learning . 10 1.4 Insights for Machine Learning and Robotics Research . 11 1.5 Statistical Machine Learning Background . 12 1.5.1 Notation and Mathematical Formalization . 12 1.5.2 Markov Property . 14 1.5.3 Markov Decision Process . 14 1.5.4 Entropy . 14 1.5.5 Kullback-Leibler (KL) Divergence . 15 1.5.6 Information and Moment Projections . 15 1.5.7 The Maximum Entropy Principle . 16 1.5.8 Background: Reinforcement Learning . 17 1.6 Formulation of the Imitation Learning Problem . 18 2 Design of Imitation Learning Algorithms 20 2.1 Design Choices for Imitation Learning Algorithms . 20 2.2 Behavioral Cloning and Inverse Reinforcement Learning 24 ii iii 2.3 Model-Free and Model-Based Imitation Learning Meth- ods ............................. 25 2.4 Observability . 28 2.4.1 Trajectories in Fully Observable Settings . 29 2.4.2 Trajectories in Partially Observable Settings . 29 2.4.3 Differences in observability between the expert and the learner . 30 2.5 Policy Representation in Imitation Learning . 31 2.5.1 Levels of Policy Abstraction . 31 2.5.2 Hierarchical vs Monolithic Policies . 33 2.5.3 Feedback vs Open-Loop/Feedback-Free Policies . 34 2.5.4 Stationarity and Stochasticity of Policies . 35 2.5.4.1 Stationary vs. Non-Stationary Policies . 36 2.5.4.2 Deterministic Policy . 36 2.5.4.3 Stochastic Policy . 37 2.6 Behavior Descriptors . 38 2.6.1 State-action Distribution . 38 2.6.2 Trajectory Feature Expectation . 38 2.6.3 Trajectory Feature Distribution . 39 2.7 Information Theoretic Understanding of Feature Matching 39 2.7.1 Information Theoretic Understanding of Imita- tion Learning Algorithms for Trajectory Learn- ing ......................... 40 2.7.2 Information Theoretic Understanding of Imita- tion Learning Algorithms in Action-State Space . 44 3 Behavioral Cloning 46 3.1 Problem Statement . 46 3.2 Design Choices for Behavioral Cloning . 48 3.2.1 Choice of Surrogate Loss Functions for Behav- ioral Cloning . 49 3.2.1.1 Quadratic Loss Function . 49 3.2.1.2 ℓ1-Loss Function . 50 3.2.1.3 Log Loss Function . 51 3.2.1.4 Hinge Loss Function . 52 3.2.1.5 Kullback-Leibler Divergence . 52 iv 3.2.2 Choice of Regression Methods for Behavioral Cloning . 53 3.3 Model-Free and Model-Based Behavioral Cloning Methods 53 3.4 Model-Free Behavioral Cloning Methods in Action-State space............................. 54 3.4.1 Model-Free Behavioral Cloning as Supervised Learning . 55 3.4.2 Imitation as Supervised Learning with Neural Networks . 56 3.4.2.1 Recent Successes of Imitation Leaning with Neural Networks . 56 3.4.2.2 Learning with Recurrent Neural Networks 58 3.4.3 Teacher-Student Interaction during Behavioral Cloning . 59 3.4.3.1 Reduction of Structured Prediction to Iterative Learning of Simple Classification 60 3.4.3.2 Confidence-Based Approach . 61 3.4.3.3 Data Aggregation Approach: DA GGER 62 3.5 Model-Free Behavioral Cloning for Learning Trajectories 65 3.5.1 Trajectory Representation . 65 3.5.1.1 Keyframe/Via-Point Based Approaches 66 3.5.1.2 Representation with Hidden Markov Models . 67 3.5.1.3 Dynamic Movement Primitives . 69 3.5.1.4 Probabilistic Movement Primitives . 73 3.5.1.5 Trajectory Representation with Time- Invariant Dynamical Systems . 75 3.5.2 Comparison of Trajectory Representations . 77 3.5.3 Generalization of Demonstrated Trajectories . 79 3.5.4 Information Theoretic Understanding of Model- FreeBC....................... 83 3.5.5 Time Alignment of Multiple Demonstrations . 84 3.5.6 Learning Coupled Movements . 86 3.5.6.1 Learning Coupled Movements with DMPs 87 v 3.5.6.2 Learning Coupled Movements with Gaussian Conditioning . 87 3.5.6.3 Learning Coupled Movements with Time-Invariant Dynamical Systems . 89 3.5.7 Incremental Trajectory Learning . 90 3.5.8 Combining Multiple Expert Policies . 93 3.6 Model-Free Behavioral Cloning for Task-Level Planning 94 3.6.1 Segmentation and Clustering for Task-Level Planning . 94 3.6.2 Learning a Sequence of Primitive Motions . 95 3.7 Model-Based Behavioral Cloning Methods . 101 3.7.1 Model-Based Behavioral Cloning Methods with Forward Dynamics Models . 101 3.7.1.1 Imitation with a Gaussian Mixture Forward Model . 103 3.7.1.2 Imitation with a Gaussian Process Forward Model . 104 3.7.2 Imitation Learning through Iterative Learning Control . 107 3.7.3 Information Theoretic Understandings of Model- Based Behavioral Cloning Methods . 108 3.8 Robot Applications with Model-Free BC Methods . 109 3.8.1 Learning to Hit a Ball with DMP . 109 3.8.2 Learning Hand-Over Tasks with ProMPs . 110 3.8.3 Learning to Tie a Knot by Modeling the Trajec- tory Distribution with Gaussian Processes . 112 3.9 Robot Applications with Model-Based BC Methods . 113 3.9.1 Learning Acrobatic Helicopter Flights . 113 3.9.2 Learning to Hit a Ball with an Underactuated Robot . 114 3.9.3 Learning to Control with DA GGER . 115 4 Inverse Reinforcement Learning 117 4.1 Problem Statement . 118 4.2 Model-Based and Model-Free IRL Methods . 120 vi 4.3 Design Choices for Inverse Reinforcement Learning Methods . 120 4.4 Model-Based Inverse Reinforcement Learning Methods . 122 4.4.1 Feature Expectation Matching . 123 4.4.2 Maximum Margin Planning . 124 4.4.3 Inverse Reinforcement Learning Based on the Maximum Entropy Principle . 126 4.4.3.1 Maximum Entropy Inverse Reinforcement Learning . 126 4.4.3.2 Maximum Causal Entropy Inverse Reinforcement Learning . 129 4.4.3.3 IRL from Failed Demonstrations . 130 4.4.3.4 Connection of Maximum Entropy Methods to Economics . 131 4.4.4 Miscellaneous Important Model-Based IRL Methods . 132 4.4.4.1 Linearly-Solvable MDPs . 132 4.4.4.2 IRL Methods Based on a Bayesian Framework . 133 4.4.5 Learning Nonlinear Reward Functions . 134 4.4.5.1 Boosting Methods . 134 4.4.5.2 Deep Network Methods . 135 4.4.5.3 Gaussian Process IRL . 135 4.4.6 Guided Cost Learning . 136 4.5 Model-Free Inverse Reinforcement Learning Methods . 138 4.5.1 Relative Entropy Inverse Reinforcement Learning 138 4.5.2 Generative Adversarial Imitation Learning . 140 4.6 Interpretation of IRL with the Maximum Entropy Prin- ciple . 141 4.7 Inverse Reinforcement Learning under Partial Observ- ability . 143 4.7.1 IRL from Partially Observable Demonstrations . 144 4.7.2 IRL with Incomplete Expert Observations . 145 4.7.3 Active Inverse Reinforcement Learning as a POMDP . 146 vii 4.7.4 Cooperative Inverse Reinforcement Learning . 146 4.8 Robot Applications with IRL Methods . 147 4.8.1 Learning to Drive a Car in a Simulator . 147 4.8.2 Learning Path Planning with MMP . 148 4.8.3 Learning Motion Planning with Deep Guided- Cost Learning . 150 4.8.4 Learning a Ball-in-a-Cup task with Relative En- tropy Inverse Reinforcement Learning . 151 5 Challenges in Imitation Learning for Robotics 152 5.1 Behavioral Cloning vs Inverse Reinforcement Learning . 152 5.2 Open Questions in Imitation Learning . 154 5.2.1 Problems Related to Demonstrated Data . 154 5.2.2 Open Questions Related to Design Choices . 155 5.2.3 Problems Related to Algorithms . 157 5.2.4 Performance Evaluation . 159 Acknowledgements 161 References 162 Abstract As robots and other intelligent agents move from simple environments and problems to more complex, unstructured settings, manually pro- gramming their behavior has become increasingly challenging and ex- pensive. Often, it is easier for a teacher to demonstrate a desired be- havior rather than attempt to manually engineer it. This process of learning from demonstrations, and the study of algorithms to do so, is called imitation learning . This work provides an introduction to imi- tation learning. It covers the underlying assumptions, approaches, and how they relate; the rich set of algorithms developed to tackle the prob- lem; and advice on effective tools and implementation. We intend this paper to serve two audiences. First, we want to famil- iarize machine learning experts with the challenges of imitation learn- ing, particularly those arising in robotics, and the interesting theoreti- cal and practical distinctions between it and more familiar frameworks like statistical supervised learning theory and reinforcement learning. Second, we want to give roboticists and experts in applied artificial in- telligence a broader appreciation for the frameworks and tools available for imitation learning. We organize our work by dividing imitation learning into directly replicating desired behavior (sometimes called behavioral cloning [Bain and Sammut, 1996]) and learning the hidden objectives of the desired behavior from demonstrations (called inverse optimal control [Kalman, 1964] or inverse reinforcement learning [Russell, 1998]). In addition to method analysis, we discuss the design decisions a practitioner must make when selecting an imitation learning approach. Moreover, appli- cation examples—such as robots that play table tennis [Kober and Peters, 2009] and programs that play the game of Go [Silver et al., 2016]— illustrate the properties and motivations behind different forms of imitation learning.
Load more

Recommended publications
  • A Imitation Learning: a Survey of Learning Methods
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by BCU Open Access A Imitation Learning: A Survey of Learning Methods Ahmed Hussein, School of Computing Science and Digital Media, Robert Gordon University Mohamed Medhat Gaber, School of Computing and Digital Technology, Birmingham City University Eyad Elyan , School of Computing Science and Digital Media, Robert Gordon University Chrisina Jayne , School of Computing Science and Digital Media, Robert Gordon University Imitation learning techniques aim to mimic human behavior in a given task. An agent (a learning machine) is trained to perform a task from demonstrations by learning a mapping between observations and actions. The idea of teaching by imitation has been around for many years, however, the field is gaining attention recently due to advances in computing and sensing as well as rising demand for intelligent applications. The paradigm of learning by imitation is gaining popularity because it facilitates teaching complex tasks with minimal expert knowledge of the tasks. Generic imitation learning methods could potentially reduce the problem of teaching a task to that of providing demonstrations; without the need for explicit programming or designing reward functions specific to the task. Modern sensors are able to collect and transmit high volumes of data rapidly, and processors with high computational power allow fast processing that maps the sensory data to actions in a timely manner. This opens the door for many potential AI applications that require real-time perception and reaction such as humanoid robots, self-driving vehicles, human computer interaction and computer games to name a few.
    [Show full text]
  • Mental States in Animals: Cognitive Ethology Jacques Vauclair
    Mental states in animals: cognitive ethology Jacques Vauclair This artHe addresses the quegtion of mentaJ states in animak as viewed in ‘cognitive ethology”. In effect, thk field of research aims at studying naturally occurring behaviours such as food caching, individual recognition, imitation, tool use and communication in wild animals, in order to seek for evidence of mental experiences, self-aw&&reness and intentional@. Cognitive ethologists use some philosophical cencepts (e.g., the ‘intentional stance’) to carry out their programme of the investigation of natural behaviours. A comparison between cognitive ethology and other approaches to the investigation of cognitive processes in animals (e.g., experimental animal psychology) helps to point out the strengths and weaknesses of cognitive ethology. Moreover, laboratory attempts to analyse experimentally Mentional behaviours such as deception, the relationship between seeing and knowing, as well as the ability of animals to monitor their own states of knowing, suggest that cognitive ethology could benefit significantly from the conceptual frameworks and methods of animal cognitive psychology. Both disciplines could, in fact, co&ribute to the understanding of which cognitive abilities are evolutionary adaptations. T he term ‘cognirive ethology’ (CE) was comed by that it advances a purposive or Intentional interpretation Griffin in The Question ofAnimd Au~aarmess’ and later de- for activities which are a mixture of some fixed genetically veloped in other publications’ ‘_ Although Griffin‘s IS’6 transmitred elements with more hexible behaviour?. book was first a strong (and certainly salutary) reacrion (Cognitive ethologists USC conceptual frameworks against the inhibitions imposed by strict behaviourism in provided by philosophers (such as rhe ‘intentional stance’)“.
    [Show full text]
  • Much Has Been Written About the Turing Test in the Last Few Years, Some of It
    1 Much has been written about the Turing Test in the last few years, some of it preposterously off the mark. People typically mis-imagine the test by orders of magnitude. This essay is an antidote, a prosthesis for the imagination, showing how huge the task posed by the Turing Test is, and hence how unlikely it is that any computer will ever pass it. It does not go far enough in the imagination-enhancement department, however, and I have updated the essay with a new postscript. Can Machines Think?1 Can machines think? This has been a conundrum for philosophers for years, but in their fascination with the pure conceptual issues they have for the most part overlooked the real social importance of the answer. It is of more than academic importance that we learn to think clearly about the actual cognitive powers of computers, for they are now being introduced into a variety of sensitive social roles, where their powers will be put to the ultimate test: In a wide variety of areas, we are on the verge of making ourselves dependent upon their cognitive powers. The cost of overestimating them could be enormous. One of the principal inventors of the computer was the great 1 Originally appeared in Shafto, M., ed., How We Know (San Francisco: Harper & Row, 1985). 2 British mathematician Alan Turing. It was he who first figured out, in highly abstract terms, how to design a programmable computing device--what we not call a universal Turing machine. All programmable computers in use today are in essence Turing machines.
    [Show full text]
  • Nonverbal Imitation Skills in Children with Specific Language Delay
    City Research Online City, University of London Institutional Repository Citation: Dohmen, A., Chiat, S. and Roy, P. (2013). Nonverbal imitation skills in children with specific language delay. Research in Developmental Disabilities, 34(10), pp. 3288-3300. doi: 10.1016/j.ridd.2013.06.004 This is the unspecified version of the paper. This version of the publication may differ from the final published version. Permanent repository link: http://openaccess.city.ac.uk/3352/ Link to published version: http://dx.doi.org/10.1016/j.ridd.2013.06.004 Copyright and reuse: City Research Online aims to make research outputs of City, University of London available to a wider audience. Copyright and Moral Rights remain with the author(s) and/or copyright holders. URLs from City Research Online may be freely distributed and linked to. City Research Online: http://openaccess.city.ac.uk/ [email protected] Full title: Nonverbal imitation skills in children with specific language delay Name and affiliations of authors Andrea Dohmen¹˒², Shula Chiat², and Penny Roy² ¹Department of Experimental Psychology, University of Oxford, 9 South Parks Road, Oxford OX1 3UD, UK ²Language and Communication Science Division, City University London, Northampton Square EC1V 0HB London, UK Corresponding author Andrea Dohmen University of Oxford / Department of Experimental Psychology 9 South Parks Road Oxford OX1 3UD, UK Email: [email protected] Telephone: 0044/1865/271334 2 Nonverbal imitation skills in children with specific language delay Abstract Research in children with language problems has focussed on verbal deficits, and we have less understanding of children‟s deficits with nonverbal sociocognitive skills which have been proposed to be important for language acquisition.
    [Show full text]
  • Deep Reinforcement Learning Via Imitation Learning
    Deep Reinforcement Learning via Imitation Learning Sergey Levine perception Action (run away) action sensorimotor loop Action (run away) End-to-end vision standard features mid-level features classifier computer (e.g. HOG) (e.g. DPM) (e.g. SVM) vision Felzenszwalb ‘08 deep learning Krizhevsky ‘12 End-to-end control standard state low-level modeling & motion motor observations estimation controller robotic prediction planning torques control (e.g. vision) (e.g. PD) deep motor sensorimotor observations torques learning indirect supervision actions have consequences Contents Imitation learning Imitation without a human Research frontiers Terminology & notation 1. run away 2. ignore 3. pet Terminology & notation 1. run away 2. ignore 3. pet Terminology & notation 1. run away 2. ignore 3. pet a bit of history… управление Lev Pontryagin Richard Bellman Contents Imitation learning Imitation without a human Research frontiers Imitation Learning training supervised data learning Images: Bojarski et al. ‘16, NVIDIA Does it work? No! Does it work? Yes! Video: Bojarski et al. ‘16, NVIDIA Why did that work? Bojarski et al. ‘16, NVIDIA Can we make it work more often? cost stability Learning from a stabilizing controller (more on this later) Can we make it work more often? Can we make it work more often? DAgger: Dataset Aggregation Ross et al. ‘11 DAgger Example Ross et al. ‘11 What’s the problem? Ross et al. ‘11 Imitation learning: recap training supervised data learning • Usually (but not always) insufficient by itself – Distribution mismatch problem • Sometimes works well – Hacks (e.g. left/right images) – Samples from a stable trajectory distribution – Add more on-policy data, e.g.
    [Show full text]
  • Imitation: Is Cognitive Neuroscience Solving the Correspondence Problem?
    ARTICLE IN PRESS TICS 384 Review TRENDS in Cognitive Sciences Vol.xx No.xx Monthxxxx Imitation: is cognitive neuroscience solving the correspondence problem? Marcel Brass1 and Cecilia Heyes2 1Department of Cognitive Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, Stephanstr. 1A, 04103 Leipzig, Germany 2Department of Psychology, University College London, Gower Street, London WC1E 6BT, UK Imitation poses a unique problem: how does the imi- framework (generalist theories) or whether it depends on tator know what pattern of motor activation will make a special purpose mechanism (specialist theories). We their action look like that of the model? Specialist then review research on the role of learning in imitation theories suggest that this correspondence problem has and observation of biological motion. Finally, we discuss a unique solution; there are functional and neurological a problem for generalist theories: If imitation depends on mechanisms dedicated to controlling imitation. General- ist theories propose that the problem is solved by general mechanisms of associative learning and action control. Box 1. Mirror neurons: What do they do? What are they for? Recent research in cognitive neuroscience, stimulated Mirror neurons in the premotor area F5 of monkeys are active both by the discovery of mirror neurons, supports generalist when the animal observes and when it executes a specific action solutions. Imitation is based on the automatic activation (for a review see [52,53]. The discovery of these cells has had a revolutionary impact, turning perception–action interaction into a of motor representations by movement observation. focus of intensive, interdisciplinary research worldwide. Naturally These externally triggered motor representations are there has been a great deal of speculation about the function of then used to reproduce the observed behaviour.
    [Show full text]
  • How to Use Outlines to Teach Rhetorical Prosody and Structure
    VANCE SCHAEFER AND LINDA ABE United States The Art of Imitation: How to Use Outlines to Teach Rhetorical Prosody and Structure onnative speakers of a language are often at a disadvantage in producing extended speech, as they have differing native (L1) N phonological systems and rhetorical traditions or little experience in giving talks. Prosody in the form of stress, rhythm, and intonation is a difficult but crucial area needed to master extended speech because prosody interacts with structure to play a key role in conveying meaning (i.e., intelligibility) and easing understanding (i.e., comprehensibility) (see Munro and Derwing 1995 for discussion on intelligibility, comprehensibility, and pronunciation). However, second-language (L2) speakers of rhetorical prosody and structure: namely, English might not effectively produce this putting the focus on using outlines. This rhetorical prosody. For example, they may article serves as a primer for practitioners not fully utilize pitch range to signal a shift of English as a second or foreign language in topics (Wennerstrom 1998), potentially (ESL/EFL), describing how to use outlines resulting in lower comprehension levels (Munro as an effective technique to teach rhetorical and Derwing 1995; Wennerstrom 1998). prosody and structure to learners at the level This difficulty appears to arise from a lack of high school, university, and beyond. This of understanding and/or effective practice, article covers the following: as gauged from our teaching experience. While the mechanics of academic writing and 1. a simple overview of prosody and grammar are taught to L2 English learners, the structure in English, described with rhetorical elements of prosody do not appear to concrete features and rules be commonly taught.
    [Show full text]
  • Learning by Imitation, Reinforcement and Verbal Rules in Problem-Solving Tasks
    Dandurand, F., Bowen, M. & Shultz, T. R. (2004). Learning by imitation, reinforcement and verbal rules in problem-solving tasks. In the Proceedings of the Third International Conference on Development and Learning (ICDL'04) Developing Social Brains. La Jolla, California, USA: The Salk Institute for Biological Studies. Learning by Imitation, Reinforcement and Verbal Rules in Problem Solving Tasks Frédéric Dandurand, Melissa Bowen, Thomas R. Shultz McGill University, Department of Psychology, 1205 Dr. Penfield Ave., Montréal, Québec, H3A 1B1, Canada, [email protected], [email protected], [email protected] Abstract ponent of intelligence. “The ability to solve problems is one of the most important manifestations of human think- Learning by imitation is a powerful process for ac- ing.” [12]. quiring new knowledge, but there has been little research Information-processing theory is currently the domi- exploring imitation’s potential in the problem solving nant approach to problem solving [12]. Problems are domain. Classical problem solving techniques tend to construed in terms of states, transitions and operators. The center around reinforcement learning, which requires essence of problem solving is a search through the state significant trial-and-error learning to reach successful space for a solution state using the operators available at goals and problem solutions. Heuristics, hints, and rea- each point while satisfying a set of problem-specific con- soning by analogy have been favored as improvements straints. over reinforcement learning, whereas imitation learning has been regarded as rote memorizing. However, re- 1.1. Learning in Problem Solving Tasks search on imitation learning in animals and infants sug- gests that what is being learned is the overall arrange- Several types of learning can occur during problem ment of actions (sequencing and planning).
    [Show full text]
  • Heterogeneity and Diffusion in the Digital Economy: Spain's Case
    Working Paper, Nº 15/29 Madrid, November 2015 Heterogeneity and diffusion in the digital economy: Spain’s case Javier Alonso Alfonso Arellano 15/29 Working Paper November 2015 Heterogeneity and diffusion in the digital economy: Spain’s case* Javier Alonso and Alfonso Arellano. Abstract The traditional Bass model (Bass, 1969) for the adoption and diffusion of new products has customarily been used to gauge the speed at which new products were adopted in a market by estimating innovation (p) and imitation (q) parameters. Rogers (2003) proposed that certain factors influence the diffusion of such products, including the educational level and age of consumers. In this article we estimate the coefficients of innovation and imitation for adopting internet, e-commerce and online banking in Spain’s case, while controlling for heterogeneity of individuals according to educational level and age. We thus find that individuals with very different p and q coefficients can coexist in one market. We then verify that the processes of an ageing population and educational improvement in Spain could give rise to long term effects on Spain’s overall innovative and imitative capacity as a result of its socio-demographic mix. Key words: Digital economy, adoption, diffusion, consumers, Spain. JEL: O30, L81, L86. *: The authors would like to thank assistants to the Conference on Digital Experimentation CODE MIT (October 16th-17th, 2015) for their constructive and useful comments. 2 / 29 www.bbvaresearch.com Working Paper November 2015 1 Introduction The innovations that have been brought in over the last few decades within the ambit of digital media have entailed the most spectacular of changes as regards both the economy and social relations.
    [Show full text]
  • Animal Culture Consequences of Sociality What Is Culture? What Is Culture? Cultural Transmission
    Animal culture Consequences of sociality Animals are exposed to behavior, sometimes novel, of others Do animals display culture? What is culture? “The totality of the mental and physical reactions and activities that characterize the behavior of individuals composing a social group collectively and individually in relations to their natural environment, to other groups, to members of the group itself and of each individual to himself” - Franz Boas (1911) “An extrasomatic (nongenetic, nonbodily), temporal continuum of things and events dependent upon symboling. Culture consists of tools, implements, utensils, clothing, ornaments, customs, institutions, beliefs, rituals, games, works of art, language, etc.” - Leslie White (~1949) What is culture? Cultural transmission “Patterns, explicit and implicit, of and for behavior acquired May occur via copying… …or via direct instruction and transmitted by symbols, constituting the distinctive achievement of human groups” - Kroeber and Kluckhohn (1952) “Learned systems of meaning, communicated by means of natural language and other symbol systems, having representational, directive, and affective functions, and capable of creating cultural entities and particular senses of reality” - Roy D'Andrade (~1984) Tool use in chimpanzees “The universal human capacity to classify, codify and communicate their experiences symbolically…a defining feature of the genus Homo” - Wikipedia (2006) Culturally transmitted behavior must persist beyond life of originator 1 Cultural transmission in macaques Cultural transmission
    [Show full text]
  • The Limits to Imitation in Rational Observational Learning∗
    The Limits to Imitation in Rational Observational Learning∗ Erik Eyster and Matthew Rabiny August 8, 2012 Abstract An extensive literature identifies how rational people who observe the behavior of other privately-informed rational people with similar tastes may come to imitate them. While most of the literature explores when and how such imitation leads to inefficient information aggregation, this paper instead explores the behavior of fully rational observational learners. In virtually any setting, they imitate only some of their predecessors, and sometimes contradict both their private information and the prevailing beliefs that they observe. In settings that allow players to extract all rele- vant information about others' private signals from their actions, we identify necessary and sufficient conditions for rational observational learning to include \anti-imitation" where, fixing other observed actions, a person regards a state of the world as less likely the more a predecessor's action indicates belief in that state. Anti-imitation arises from players' need to subtract off the sources of correlation in others' actions, and is mandated by rationality in settings where players observe many predecessors' actions ∗We thank Min Zhang for excellent research assistance as well as Paul Heidhues and seminar participants at Berkeley, DIW Berlin, Hebrew University, ITAM, LSE, Penn, Tel Aviv University, and WZB for their comments. yEyster: Department of Economics, LSE, Houghton Street, London WC2A 2AE, UK. Rabin: Department of Economics, UC Berkeley, 549 Evans Hall #3880, Berkeley, CA 94720-3880, USA. 1 but not all recent or concurrent ones. Moreover, in these settings, there is always a positive probability that some player plays contrary to both her private information and the beliefs of every single person whose action she observes.
    [Show full text]
  • Innovation Has Become the Catchword of the Twentieth and the Twentieth
    Innovation and Imitation: Why is Imitation not Innovation? Benoît Godin 385 rue Sherbrooke Est Montréal, Québec Canada H2X 1E3 Project on the Intellectual History of Innovation Working Paper No. 25 2016 Previous Papers in the Series: 1. B. Godin, Innovation: the History of a Category. 2. B. Godin, In the Shadow of Schumpeter: W. Rupert Maclaurin and the Study of Technological Innovation. 3. B. Godin, The Linear Model of Innovation (II): Maurice Holland and the Research Cycle. 4. B. Godin, National Innovation System (II): Industrialists and the Origins of an Idea. 5. B. Godin, Innovation without the Word: William F. Ogburn’s Contribution to Technological Innovation Studies. 6. B. Godin, ‘Meddle Not with Them that Are Given to Change’: Innovation as Evil. 7. B. Godin, Innovation Studies: the Invention of a Specialty (Part I). 8. B. Godin, Innovation Studies: the Invention of a Specialty (Part II). 9. B. Godin, καινοτομία: An Old Word for a New World, or the De-Contestation of a Political and Contested Concept. 10. B. Godin, Innovation and Politics: The Controversy on Republicanism in Seventeenth Century England. 11. B. Godin, Social Innovation: Utopias of Innovation from circa-1830 to the Present. 12. B. Godin and P. Lucier, Innovation and Conceptual Innovation in Ancient Greece. 13. B. Godin and J. Lane, ‘Pushes and Pulls”: The Hi(S)tory of the Demand Pull Model of Innovation. 14. B. Godin, Innovation after the French Revolution, or, Innovation Transformed: From Word to Concept. 15. B. Godin, Invention, Diffusion and Innovation. 16. B. Godin, Innovation and Science: When Science Had Nothing to Do with Innovation, and Vice-Versa.
    [Show full text]