Functional Autonomy Techniques for Manipulation in Uncertain Environments Thesis by Joseph Bowkett In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy CALIFORNIA INSTITUTE OF TECHNOLOGY Pasadena, California 2020 Defended March 13, 2020 ii © 2020 Joseph Bowkett ORCID: 0000-0002-3101-489X All rights reserved except where otherwise noted iii ACKNOWLEDGEMENTS First and foremost, my thanks go to Joel, who numbers among the most conscientious and affable humans I have met on this planet. His patience and good-natured guidance made the graduate school experience infinitely more tolerable than it would otherwise have been. I am eternally indebted to many of the fine folk at the Jet Propulsion Laboratory, who saw fit to offer me work, technical instruction, and social diversion. In particular, the denizens of the 198-B7 lab, both past and present, taught me more about robotics than I believe I could have learnt in any other forum. With any luck I’ll be able to repay that debt in-kind after starting there as an employee. Much of my work undertaken through JPL revolved around the Robotics Collabo- rative Technology Alliance, and across 7 integration meetings in a range of cities I was incredibly fortunate to have the opportunity to work alongside many superb engineers, roboticists, and researchers from General Dynamics, the Army Research Laboratory, the University of Washington, Carnegie Mellon University, the Univer- sity of Pennsylvania, and many others. Despite occasionally trying circumstances, they always managed to maintain a cheery outlook, and somehow made the drudge of implementing a million software fixes across more than a million lines of code an enjoyable experience. Of course, I never would have made it to Caltech in the first place were it not for the unwavering support of my family and friends in New Zealand. In particular, this thesis is dedicated to my parents, for their dedication to ensuring I had all the educational resources and anything else I could want for to pursue a fulfilling career and life, even if it meant flying halfway around the world to see me. The first year of classes at Caltech was easily the most challenging of my life, but the comradery and collaboration of my 2014 intake classmates of the MCE and GALCIT departments made it survivable, nigh on enjoyable. The many late nights spent in various libraries or the Keith-Spalding building finishing problem sets while eating 3am orders of thai food will be one of my most enduring memories of Caltech. Orientation week of that first year, around a poker table and otherwise, is also when I met many of the sterling individuals with whom I spent my time outside of studies. Among them, my thanks go to the Fighting Pinecones, skiing, and squash buddies, for managing to get me up and out of the lab every once in a while. The most special iv of mentions must go to the Fremont lads whom, with the ever available supply of star tangled banglers, witty banter, and debate on any topic under the Sun (or beyond it), proved the perfect cocktail for keeping this stressed grad student sane. Thanks also to the MCE & JPL administrative staff, whose guidance and patience dealing with my frequent work trips for integration meetings and conferences cer- tainly saved the logistical day numerous times, and allowed the grad school process to pass as smoothly as it could have. Last but not least, the PhD experience would not be complete without the amazing lab mates I had the pleasure of working with, from both the Burdick and Ames groups. The many diversions discussing geopolitical affairs, the minutia of programming languages, and the merits of different text editors or desk heights, served as welcome stimulation when work, at times, proved monotonous or stressful. Funding Sources Research relating to Chapter 3 of this thesis was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration (80NM0018D004). Work on Chapters 4 and 5 was sponsored by the Army Research Laboratory and was accomplished under Cooperative Agreement Number W911NF-10-2-0016. Government sponsorship acknowledged. v ABSTRACT As robotic platforms are put to work in an ever more diverse array of environments, their ability to deploy visuomotor capabilities without supervision is complicated by the potential for unforeseen operating conditions. This is a particular challenge within the domain of manipulation, where significant geometric, semantic, and kinetic understanding across the space of possible manipulands is necessary to allow effective interaction. To facilitate adoption of robotic platforms in such environments, this work investigates the application of functional, or behavior level, autonomy to the task of manipulation in uncertain environments. Three functional autonomy techniques are presented to address subproblems within the domain. The task of reactive selection between a set of actions that incur a probabilistic cost to advance the same goal metric in the presence of an operator action preference is formulated as the Obedient Multi-Armed Bandit (OMAB) problem, under the purview of Reinforcement Learning. A policy for the problem is presented and evaluated against a novel performance metric, disappointment (analogous to pro- totypical MAB’s regret), in comparison to adaptations of existing MAB policies. This is posed for both stationary and non-stationary cost distributions, within the context of two example planetary exploration applications of multi-modal mobility, and surface excavation. Second, a computational model that derives semantic meaning from the outcome of manipulation tasks is developed, which leverages physics simulation and clustering to learn symbolic failure modes. A deep network extracts visual signatures for each mode that may then guide failure recovery. The model is demonstrated through application to the archetypal manipulation task of placing objects into a container, as well as stacking of cuboids, and evaluated against both synthetic verification sets and real depth images. Third, an approach is presented for visual estimation of the minimum magnitude grasping wrench necessary to extract massive objects from an unstructured pile, subject to a given end effector’s grasping limits, that is formulated for each object as a “wrench space stiction manifold”. Properties are estimated from segmented RGBD point clouds, and a geometric adjacency graph used to infer incident wrenches upon each object, allowing candidate extraction object/force-vector pairs to be selected from the pile that are likely to be within the system’s capability. vi PUBLISHED CONTENT AND CONTRIBUTIONS Joseph Bowkett, Matt Burkhardt, and Joel W. Burdick (2016). “Combined Energy Harvesting and Control of Moball: A Barycentric Spherical Robot”. 2016 In- ternational Symposium on Experimental Robotics. In: Springer Proceedings in Advanced Robotics, vol 1. pp. 71–83. doi: 10.1007/978-3-319-50115-4_7. J.B. built the test apparatus, undertook harvesting experiments to validate prior simulations. Also designed, simulated, and experimentally tested strategies for magnet control. Not included in thesis as is outside the functional autonomy narrative. Joseph Bowkett and Rudranarayan Mukherjee (2017). “Comparison of control methods for two-link planar flexible manipulator”. 2017 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference In: 13th International Conference on Multibody Systems, Nonlinear Dynamics, and Control doi: 10.1115/DETC2017-67937. J.B. formulated reduced dynamics for the problem, designed and implemented the two control schemes, tested them in simulation, then demonstrated their use on a purpose built experimental platform. Not included in thesis as is outside the functional autonomy narrative. Joseph Bowkett, Renaud Detry, and Larry H. Matthies (2018). “Semantic Under- standing of Task Outcomes: Visually Identifying Failure Modes Autonomously Discovered in Simulation”. 2018 IEEE International Conference on Robotics and Automation Workshop: Multimodal Robot Perception: Representing a Complex World url: https://natanaso.github.io/rcw-icra18/. J.B. developed the task space formulation, designed and scripted task simulations with depth map generation and processing, then constructed, trained and eval- uated the visual classification model. Constitutes the early portions of Chapter 4. Luca Massari, Calogero M. Oddo, Edoardo Sinibaldi, Renaud Detry, Joseph Bowkett, and Kalind C. Carpenter (2019). “Tactile sensing and control of robotic manipu- lator integrating fiber bragg grating strain-sensor”. In: Frontiers in Neurorobotics vol 13. doi: 10.3389/fnbot.2019.00008. J.B. aided in the analysis of test data as it pertained to grasping. Paper provides a motivating argument for the inclusion of proprioceptive inference in manipuland comprehension, referenced in Chapter 5. William Reid, Gareth Meirion-Griffith, Sisir Karumanchi, Blair Emanuel, Brendan Chamberlain-Simon, Joseph Bowkett, and Michael Garrett (2019). “Actively Articulated Wheel-on-Limb Mobility for Traversing Europa Analogue Terrain”. In: 12th Conference on Field and Service Robotics vol 6. vii J.B. aided in maintenance and upgrade of experimental platform, as well as some lab experiments. Multiple mobility mode demonstrated in paper are a motivating example for algorithm described in Chapter 3. Joseph Bowkett, Joel W. Burdick, and Renaud Detry (2019). “Visual Extraction Ef- fort Estimation