University of Massachusetts Amherst ScholarWorks@UMass Amherst Open Access Dissertations 2-2011 A Teleological Approach to Robot Programming by Demonstration John Douglas Sweeney University of Massachusetts Amherst Follow this and additional works at: https://scholarworks.umass.edu/open_access_dissertations Part of the Computer Sciences Commons Recommended Citation Sweeney, John Douglas, "A Teleological Approach to Robot Programming by Demonstration" (2011). Open Access Dissertations. 351. https://scholarworks.umass.edu/open_access_dissertations/351 This Open Access Dissertation is brought to you for free and open access by ScholarWorks@UMass Amherst. It has been accepted for inclusion in Open Access Dissertations by an authorized administrator of ScholarWorks@UMass Amherst. For more information, please contact [email protected]. A TELEOLOGICAL APPROACH TO ROBOT PROGRAMMING BY DEMONSTRATION A Dissertation Presented by JOHN D. SWEENEY Submitted to the Graduate School of the University of Massachusetts Amherst in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY February 2011 Computer Science c Copyright by John D. Sweeney 2011 All Rights Reserved A TELEOLOGICAL APPROACH TO ROBOT PROGRAMMING BY DEMONSTRATION A Dissertation Presented by JOHN D. SWEENEY Approved as to style and content by: Roderic A. Grupen, Chair Oliver Brock, Member Andrew H. Fagg, Member Rachel Keen, Member Andrew G. Barto, Department Chair Computer Science To my parents. ACKNOWLEDGMENTS The help of many people was instrumental in creating this dissertation. Foremost is my advisor, Rod Grupen. I am very grateful for all of the guidance and support he has given me throughout my graduate studies. I have learned so much from his insight and will always appreciate his consideration and candor. Rod has been a wonderful mentor, showing me how to do meaningful research in robotics, peel back the frontier of knowledge, and make robots do cool things. His warm and friendly manner set the tone for the lab, and helped make it such an enjoyable place to work and learn. For all of this I am truly thankful. I would also like to thank the members of my committee for their help and advice throughout this journey. I would like to thank Oliver Brock for always keeping his office door open to me; I thoroughly enjoyed our many conversations. Oliver is always eager to share his enthusiasm and deep knowledge of robotics and he introduced me to many new areas of the field, for which I am grateful. I am also indebted to Andy Fagg for introducing me to Dexter, and mentoring my first work on the robot. It was from our initial collaboration that the work in dissertation was born. I greatly respect and admire Andy’s breadth of knowledge and technical abilities, both in and out of robotics. Thanks to Rachel Keen for her insightful perspective on the intersection between robotics and developmental psychology. Bouncing ideas off her and the other members of her lab was always a fruitful exercise. v I also owe a huge debt of gratitude to all the members of the Laboratory for Perceptual Robotics, past and present, that I had the pleasure of working with, and that helped shape the course of my studies: TJ Brunette, Patrick Deegan, Stephen Hart, Emily Horrell, Chao Ou, Rob Platt, Chandu Ravela, Shiraj Sen, and Bryan Thibodeau. A special thanks to Michael Rosenstein for his advice and for sharing the helpful teleoperation software libraries he developed for Dexter. I am truly fortunate to have been able to pursue a degree in robotics at this univer- sity. The Computer Science department is filled with interesting people doing great research. My friends in the department have made this time in Amherst so enjoyable and rewarding, for which I am immensely thankful. I would like to thank my globe-trotting brother, Tim, for his support and enthusiasm for my work throughout this process. His music provided the soundtrack for many late nights working in the lab. I would also like to thank my parents for their love and support, and for providing me with the opportunity to pursue my interests, academic and otherwise. For this I am eternally grateful. I would especially like to thank my wife Casey, for her never-ending support of me in this endeavor. Without her love, encouragement and understanding, I would never have been able to complete this work. All of my love to you. vi ABSTRACT A TELEOLOGICAL APPROACH TO ROBOT PROGRAMMING BY DEMONSTRATION FEBRUARY 2011 JOHN D. SWEENEY B.Sc., CARNEGIE MELLON UNIVERSITY M.Sc., UNIVERSITY OF MASSACHUSETTS AMHERST Ph.D., UNIVERSITY OF MASSACHUSETTS AMHERST Directed by: Professor Roderic A. Grupen This dissertation presents an approach to robot programming by demonstration based on two key concepts: demonstrator intent is the most meaningful signal that the robot can observe, and the robot should have a basic level of behavioral competency from which to interpret observed actions. Intent is a teleological, robust teaching signal invariant to many common sources of noise in training. The robot can use the knowledge encapsulated in sensorimotor schemas to interpret the demonstration. vii Furthermore, knowledge gained in prior demonstrations can be applied to future sessions. I argue that programming by demonstration be organized into declarative and pro- cedural components. The declarative component represents a reusable outline of underlying behavior that can be applied to many different contexts. The procedu- ral component represents the dynamic portion of the task that is based on features observed at run time. I describe how statistical models, and Bayesian methods in particular, can be used to model these components. These models have many fea- tures that are beneficial for learning in this domain, such as tolerance for uncertainty, and the ability to incorporate prior knowledge into inferences. I demonstrate this architecture through experiments on a bimanual humanoid robot using tasks from the pick and place domain. Additionally, I develop and experimentally validate a model for generating grasp candidates using visual features that is learned from demonstration data. This model is especially useful in the context of pick and place tasks. viii TABLE OF CONTENTS Page ACKNOWLEDGMENTS ............................................. v ABSTRACT ................................................... ...... vii LIST OF TABLES ................................................... xv LIST OF FIGURES.................................................. xvi CHAPTER 1. INTRODUCTION ................................................. 1 1.1 Research Approach ................................ ...............3 1.2 Contributions ................................... .................6 2. RELATED WORK ................................................ 8 2.1 Methods for Acquiring Training Data . ..............9 2.2 Abstraction in PbD Systems ......................... .............11 2.2.1 Procedural Abstraction ......................... ...........12 ix 2.2.2 Declarative Abstraction ........................ ............17 2.3 The Teleological Approach......................... ...............20 2.3.1 Goals......................................... ...........22 2.3.2 Means ......................................... ..........24 2.3.3 Constraints................................... ............24 2.3.4 Action Selection............................... ............25 2.4 Biological Motivation ............................ ................26 3. THE CONTROL BASIS AND SENSORIMOTOR SCHEMAS ..... 29 3.1 TheControlBasis ................................. ..............30 3.1.1 Artificial Potential Approaches in Robotics . ...........35 3.1.2 Multi-Objective Control........................ ............36 3.1.3 Primitive Controllers .......................... ............37 3.1.3.1 A Controller for Reaching ...................... ....37 3.1.3.2 A Controller for Manipulability . .....39 3.1.3.3 A Controller for Grasping ...................... ....40 3.2 Turning Controllers into Behavior . ................40 3.2.1 Discrete State Dynamics ......................... ..........41 3.2.2 Sensorimotor Behaviors ......................... ...........43 3.2.2.1 Search-Track ................................ .....44 x 3.2.3 Hierarchical Structure ......................... ............46 3.2.3.1 Reach-Grasp ................................. ....46 3.3 Representing Complex Behavior with Schemas . .............48 3.3.1 Declarative and Procedural Decomposition of Controllers .......48 3.3.2 Sensorimotor Schemas ........................... ..........50 3.3.3 Pick-And-Place ................................ ...........51 3.4 Discussion...................................... ................53 4. A DECLARATIVE REPRESENTATION FOR PROGRAMMING BY DEMONSTRATION ................... 54 4.1 Declarative Representation of Schemas . ...............55 4.1.1 Hidden Markov Models ............................ ........56 4.2 Interpreting Demonstration ....................... ................61 4.3 Applying Knowledge from Demonstration . .............66 4.3.1 Experimental Apparatus ......................... ..........66 4.3.2 Experiment: A sorting task....................... ..........68 4.3.3 Discussion.................................... ............73 5. PROCEDURAL INFORMATION FROM DEMONSTRATION ........................................... 74 5.1 Generative Models for Procedural Policies. ................76 xi 5.1.1 A Generative Model for Pick and Place Proceduralization.......80 5.1.2 Experiments with a Sorting Task