The TidyUpRobot Project

Wolfram Burgard, Bernhard Nebel, Martin Riedmiller, Jürgen Sturm, Felix Endres, Jürgen Hess, Bastian Steder, Christian Dornhege, Sascha Lange, Stefan Welker

Department of Computer Science , University of Freiburg, Germany

 Freiburg has 200.000 inhabitants  Freiburg University has 30.000 students  Traditional university with a technical faculty (CS + MST)  Strong in AI and Robotics and AI in Freiburg Autonomous Intelligent Systems Wolfram Burgard

Foundations of Bernhard Nebel

Machine Learning Martin Riedmiller

Humanoid Maren Bennewitz Social Robotics Kai Arras Autonomous Intelligent Systems

 Mobile robots  Mapping  Localization  Navigation  Probabilistic robotics  Multi systems  Embedded systems Foundations of Artificial Intelligence

 Action planning  Theory and practice  Planning competitions  Qualitative temporalspatial reasoning  RoboCup soccer  World champion three times  Autonomous table soccer  Robocup Rescue League Machine Learning

 Reinforcement learning  Neural networks  Fast, efficient learning  Neural controllers  Applications:  Forecasting systems  RoboCup soccer  Industry Project Goal of TidyUpRobot

 Reliable execution of fetchandcarry tasks  Mobile manipulation  Domestic environments  Example: Clear table in the living room, place objects in cabinets where they belong Module Overview

 Reuseable software modules  Licensed under (L)GPL

Feature-based Symbolic Planner With Object Recognition Semantic Attachments

Generalized Map Representation

Kinematic Models for Reinforcement Learning Articulated Objects Motion Controller Module: Object Recognition

 Learn object models  Detect and localize objects in point clouds Module: Object Recognition Approach:  Match point features from models  Estimate object candidate pose  Score and filter candidates Module: Articulated Objects

 Detect articulated objects  Learn kinematic models  Open doors and drawers  Use cupboards to stow objects Module: Generalized Mapping

 Topological maps (multifloor)  Augmented with semantic information  Detected (articulated) objects  Kinematic models  Action/skill models  Symbolic description (for planner) Module: Semantic Planning

 Plan with incomplete knowledge  Sensory actions that gain knowledge  Assertions (pre/postconditions)  Continuous replanning, triggered by assertions Module: Semantic Planning

 Problem: Not all realworld facts can be represented efficiently on the logical level  Solution: Semantic attachments (use the specific algorithms, e.g., geometric path planner) Module: Action/Skill Learning

 Learning from experience: Reinforcement Learning for optimizing initial trajectories  Improve available actions  Acquire and provide new actions  Identify constraints to be integrated as new semantic attachments Project Evaluation

 Task execution reliability (>90%)  Repeatability at three other PR2 sites (≥66%)  Reusability of our code (three other groups, using at least one module) Open-Source Contributions

 Graphbased map representation (LGPL)  3D object detection and localization (LGPL)  Articulated Objects (LGPL)  Symbolic planner (GPL)  Manipulation skill learner (LGPL)

 http://code.google.com/p/alufr-ros-pkg Conclusions

 Joint initiative of the AI and robotics groups at the University of Freiburg  TidyUpRobot Project  Reliable execution of fetchandcarry tasks  Looking for labs interested in using our modules/application