A system architecture for learning robots Adam Leigh Haber A thesis in fulfilment of the requirements for the degree of Doctor of Philosophy, October 2014. School of Computer Science and Engineering Faculty of Engineering PLEASE TYPE THE UNIVERSITY OF NEW SOUTH WALES Thesis/Dissertation Sheet Surname or Family name: Haber First name: Adam Other name/s: Leigh Abbreviation for degree as given in the University calendar: PhD School: Computer Science & Engineering (CSE) Faculty: Engineering Title: A system architecture for learning robots Abstract 350 words maximum: (PLEASE TYPE) Control systems for intelligent, autonomous robots require the synthesis of a varied set of components in a highly constrained manner. Individual algorithmic solutions for many necessary components are now firmly in place. However, the mechanisms and dynamics of integration into complete systems have received less attention. Robot architectures provide principled frameworks to guide this process. This thesis presents a novel robot architecture, called MALA, constructed specifically to incorporate the diversity of specialised components necessary for autonomous robots, and integrate them with planning systems necessary for flexible reasoning. The extent to which robots can function autonomously over long timescales is dependent on their ability to learn from experience. Therefore, MALA is also focused on embedding learning into all aspects of robot software. Previous research has typically focused on robot learning an isolated setting, rather than interleaved with extended autonomous operation. Integrated approaches such as MALA enable robot learning to be analysed in this embedded context. To demonstrate this, MALA is used to control a simulated rescue robot that explores a disaster site, constructs plans to achieve its goals, and simultaneously learns to drive under difficult conditions. It is shown that the learning components enable the robot’s performance to consistently improve. There is significant potential to improve robot performance by enabling learning processes operating at different levels of abstraction to influence and guide each other, which is referred to as hybrid learning. A particularly novel aspect of MALA is the emphasis placed on hybrid learning in its design. Using MALA, a hybrid learning system capable of simultaneously acquiring task- and control- level knowledge from demonstrations is introduced. A robot initially acquires relational models of novel actions through Explanation-Based Learning (EBL), and control-level knowledge of how to perform these actions using Dynamic Motion Primitives (DMPs). It then generalises these action models using an active learning algorithm based on Inductive Logic Programming (ILP). The system is shown to be capable of learning complex manipulation actions involving both dynamic and logical characteristics, such as hitting with a hammer, and using these actions to make plans to achieve difficult goals. Declaration relating to disposition of project thesis/dissertation I hereby grant to the University of New South Wales or its agents the right to archive and to make available my thesis or dissertation in whole or in part in the University libraries in all forms of media, now or here after known, subject to the provisions of the Copyright Act 1968. I retain all property rights, such as patent rights. I also retain the right to use in future works (such as articles or books) all or part of this thesis or dissertation. I also authorise University Microfilms to use the 350 word abstract of my thesis in Dissertation Abstracts International (this is applicable to doctoral theses only). …………………………………………………………… ……………………………………..……………… ……….……………………...…….… Signature Witness Date The University recognises that there may be exceptional circumstances requiring restrictions on copying or conditions on use. Requests for restriction for a period of up to 2 years must be made in writing. Requests for a longer period of restriction may be considered in exceptional circumstances and require the approval of the Dean of Graduate Research. FOR OFFICE USE ONLY Date of completion of requirements for Award: THIS SHEET IS TO BE GLUED TO THE INSIDE FRONT COVER OF THE THESIS Originality Statement I hereby declare that this submission is my own work and to the best of my knowl- edge it contains no materials previously published or written by another person, or substantial proportions of material which have been accepted for the award of any other degree or diploma at UNSW or any other educational institution, except where due acknowledgement is made in the thesis. Any contribution made to the research by others, with whom I have worked at UNSW or elsewhere, is explicitly acknowledged in the thesis. I also declare that the intellectual content of this thesis is the product of my own work, except to the extent that assistance from others in the project’s design and conception or in style, presentation and linguistic expression is acknowledged. Signed Date iii COPYRIGHT STATEMENT ‘I hereby grant the University of New South Wales or its agents the right to archive and to make available my thesis or dissertation in whole or part in the University libraries in all forms of media, now or here after known, subject to the provisions of the Copyright Act 1968. I retain all proprietary rights, such as patent rights. I also retain the right to use in future works (such as articles or books) all or part of this thesis or dissertation. I also authorise University Microfilms to use the 350 word abstract of my thesis in Dissertation Abstract International (this is applicable to doctoral theses only). I have either used no substantial portions of copyright material in my thesis or I have obtained permission to use copyright material; where permission has not been granted I have applied/will apply for a partial restriction of the digital copy of my thesis or dissertation.' Signed ……………………………………………........................... Date ……………………………………………........................... AUTHENTICITY STATEMENT ‘I certify that the Library deposit digital copy is a direct equivalent of the final officially approved version of my thesis. No emendation of content has occurred and if there are any minor variations in formatting, they are the result of the conversion to digital format.’ Signed ……………………………………………........................... Date ……………………………………………........................... INCLUSION OF PUBLICATIONS STATEMENT UNSW is supportive of candidates publishing their research results during their candidature as detailed in the UNSW Thesis Examination Procedure. Publications can be used in their thesis in lieu of a Chapter if: • The student contributed greater than 50% of the content in the publication and is the “primary author”, ie. the student was responsible primarily for the planning, execution and preparation of the work for publication • The student has approval to include the publication in their thesis in lieu of a Chapter from their supervisor and Postgraduate Coordinator. • The publication is not subject to any obligations or contractual agreements with a third party that would constrain its inclusion in the thesis Please indicate whether this thesis contains published material or not. This thesis contains no publications, either published or submitted for publication ☐ (if this box is checked, you may delete all the material on page 2) Some of the work described in this thesis has been published and it has been documented in the relevant Chapters with acknowledgement (if this box is ☒ checked, you may delete all the material on page 2) This thesis has publications (either published or submitted for publication) ☐ incorporated into it in lieu of a chapter and the details are presented below CANDIDATE’S DECLARATION I declare that: • I have complied with the Thesis Examination Procedure • where I have used a publication in lieu of a Chapter, the listed publication(s) below meet(s) the requirements to be included in the thesis. Name Signature Date (dd/mm/yy) Adam Leigh Haber 24/09/2019 Postgraduate Coordinator’s Declaration (to be filled in where publications are used in lieu of Chapters) I declare that: • the information below is accurate • where listed publication(s) have been used in lieu of Chapter(s), their use complies with the Thesis Examination Procedure • the minimum requirements for the format of the thesis have been met. PGC’s Name PGC’s Signature Date (dd/mm/yy) i Abstract Control systems for intelligent, autonomous robots require the synthesis of a diverse set of components in a highly constrained manner. Algorithmic solutions for many necessary individual modules are now firmly in place. However, the mechanisms and dynamics of integration into complete systems have received less attention. Robot architectures provide principled frameworks to guide this process. This thesis presents a novel robot architecture, called MALA, constructed specifically to incorporate the diversity of specialised components necessary for autonomous robots, and integrate them with planning systems necessary for flexible reasoning. The extent to which robots can function autonomously over long timescales is dependent on their ability to learn from experience. Therefore, MALA is also focused on embedding learning into all aspects of robot software. Previous research has typically focused on robot learning an isolated setting, rather than interleaved with extended autonomous operation. Integrated approaches such as MALA enable robot learning to be analysed in this