Action Selection in Cooperative Robot Soccer Using Case-Based Reasoning
Total Page:16
File Type:pdf, Size:1020Kb
Action Selection in Cooperative Robot Soccer using Case-Based Reasoning Tesi Doctoral Autora Raquel Ros Espinoza Director Ramon L´opez de M`antaras Tutor Josep Puyol Gruart Programa de Doctorat en Inform`atica Departament de Ci`encies de la Computaci´o Universitat Aut`onoma de Barcelona Realitzada a Institut d’Investigaci´oen Intel.lig`encia Artificial Consejo Superior de Investigaciones Cient´ıficas Bellaterra, Febrer del 2008. al Pepe y a la Mazo iv Acknowledgments Como se suele hacer en estos casos, tendr´ıa que comenzar agradeciendo a mi director de tesis, a Ramon. Y de hecho, lo har´e. Pero no solo por haberme guiado y dado consejo en mi investigaci´on durante estos ´ultimos cuatro a˜nos, sino tambi´en por consentirme y permitirme el cambio de tema tesis el d´ıa que se lo propuse, lo cual adem´as supon´ıa la adquisici´on de nuevos robots. Todo hay que decirlo, y para suerte m´ıa, a ´el tambi´en le hac´ıan gracia estos juguetes, y no fue dif´ıcil convencerlo de este cambio, que finalmente se presenta hoy en esta tesis. Junto a ´el, y aunque a efectos burocr´aticos no figura formalmente, Josep Llu´ıs ha sido mi segundo director. Sin duda alguna las discusiones y reuniones con ´el han sido de gran utilidad a la hora de desarrollar esta tesis, y es por ello que parte de este trabajo se lo debo a ´el. Continuar´eagradeciendo el apoyo de otro “jefe”, aunque un poco m´as lejano a la tesis, que me abri´olas puertas del instituto cuando yo empezaba a descubrir el ´area de la inteligencia artificial. Carles, mi primer director en el mundo de la investigaci´on, con quien tengo garantizado pasar buenos momentos charlando, riendo... y comiendo en la mas´ıa! I sincerely thank Manuela Veloso, without whom this thesis wouldn’t have even begun. She not only accepted to work with me proposing new investi- gation trends in my career, but always enthusiastically received me in her lab in several occasions giving me the oppotunity of living new experiencies out of the IIIA. Also thanks to the CMDash team, specially Sonia, Doug and Colin, with whom I’m happy to say that besides sharing research interests, we also share a close friendship. And before flying back to Spain to continue with the acknowledgments, I would like to thank Cindy Marling, for reviewing this dissertation, as well as for her helpful comments and suggestions. Agradezco a la gente del IIIA en general, por los buenos momentos com- partidos a la hora del caf´e, las comidas, los viajes y congresos. Especialmente a amigos m´as cercanos, Eva, Maarten, Dani y Jordi, quienes me han ayudado de una manera u otra y gracias a quienes incluso los d´ıas m´as duros han sido m´as f´aciles de superar. A mis amigos de la uni, Enric, Miquel y Pep, a “les nenes”, Cris, Meri i Glor, a mi familia, Saray, D´ıdac (petit), Beto y Asun, y a todos los amigos que a pesar de no entender muy bien mi trabajo, han estado apoy´andome en todo momento e intentando comprender lo que hago con unos perritos futboleros! Evidentemente dejo al final a los “m´as importantes”! A D´ıdac, por acom- pa˜narme desde el primer d´ıa (nunca mejor dicho!) en este proyecto, no s´olo a nivel profesional discutiendo ideas, programando y escribiendo papers, pero sobretodo apoy´andome a nivel personal, en los buenos momentos y en los de v vi frustraci´on, siempre con optimismo y confiando en mi capacidad para salir adelante. Finalmente, a los protagonistas de esta tesis, a mis robots: Nata, Boira, Fang y Terra, sin quienes, definitivamente, esta tesis no hubiera podido ser completada! Raquel Ros holds a scholarship from the Generalitat de Catalunya Govern- ment. This work has been partially funded by the Spanish Ministry of Educa- tion and Science project MID-CBR (TIN2006-15140-C03-01) and by the Gener- alitat de Catalunya under the grant 2005-SGR-00093. Abstract Designing the decision-making engine of a team of robots is a challenging task, not only due to the complexity of the environment where the robots usually perform their task, which include uncertainty, dynamism and imprecision, but also because the coordination of the team must be included in this design. The robots must be aware of other robots’ actions to cooperate and to successfully achieve their common goal. Besides, decisions must be made in real-time and with limited computational resources. This thesis contributes a novel case-based approach for action selection and coordination in joint multi-robot tasks in real environments. This approach has been applied and evaluated in the representative domain of robot soccer, although the ideas presented are applicable to domains such as disaster rescue operations, exploration of unknown environments and underwater surveil- lance, among others. The retrieval process proposes a case to reuse, evaluating the candidate cases through different measures to overcome the real world characteristics, including the adversarial component which is a key ingredient in the robot soccer domain. Unlike classical case-based reasoning engines, the case reuse consists in the execution of a set of actions through a team of robots. There- fore, from the multi-robot perspective, the system has to include a mechanism for deciding who does what and how. In this thesis, we propose a multi-robot architecture along with a coordination mechanism to address these issues. We have validated the approach experimentally both in a simulated envi- ronment and with real robots. The results showed that our approach achieves the expected goals of the thesis, i.e. designing the behavior of a cooperative team of robots. Moreover, the experimentation also showed the advantages of using collaborative strategies in contrast to individualistic ones, where the adversarial component plays an important role. vii viii Contents Contents ix 1 Introduction 1 1.1 MotivationandOverview . 3 1.2 ProblemDomain ........................... 7 1.3 Contributions ............................. 10 1.4 Publications .............................. 11 1.5 OutlineoftheThesis ......................... 11 2 CBRPreliminariesandRelatedWork 13 2.1 CaseBasedReasoning . .... .... .... ... .... .... 13 2.2 CBRAppliedtoRoboCup . 16 2.2.1 Karoletal............................ 16 2.2.2 Lin,ChenandLiu ...................... 16 2.2.3 BergerandL¨ammel. 17 2.2.4 Wendleret.al ......................... 17 2.2.5 Marlingetal. ......................... 18 2.2.6 Ahmadietal. ......................... 19 2.2.7 Steffens............................. 19 2.3 OtherModelsAppliedtoRoboCup. 20 2.3.1 Learning from Observation or Imitation . 20 2.3.2 ReinforcementLearning . 20 2.3.3 PatternRecognition . 22 2.3.4 FuzzyTheory ......................... 22 2.3.5 Planning............................ 23 2.3.6 NeuralNetworks .... .... .... ... .... .... 24 2.3.7 EvolutionaryAlgorithms . 25 2.3.8 OtherApproaches ... .... .... ... .... .... 26 2.4 CBR Applied to Other Robotic-Related Domains . 27 2.5 Summary................................ 29 3 The Retrieval Step 33 3.1 CaseDefinition ............................ 33 3.1.1 ProblemDescription . 33 3.1.2 SolutionDescription . 35 3.1.3 CaseScopeRepresentation . 37 3.1.4 CaseExample......................... 38 3.2 CaseBaseDescription . .... .... .... ... .... .... 39 ix x CONTENTS 3.3 CaseRetrieval ............................. 41 3.3.1 SimilarityMeasure . 41 3.3.2 CostMeasure ......................... 44 3.3.3 CaseApplicabilityMeasure . 47 3.3.4 CaseFiltering ......................... 50 3.3.5 Experiments.......................... 53 3.4 ConclusionsandFutureWork . 59 4 CaseReusethroughaMulti-RobotSystem 61 4.1 RobotArchitecture .......................... 61 4.1.1 DeliberativeSystem . 63 4.1.2 ExecutiveSystem... ... .... .... .... ... .. 64 4.2 Multi-RobotSystemandCaseReuse . 65 4.3 ConclusionsandFutureWork . 69 5 Learning the Scopes of Cases 71 5.1 ScopeAdaptationAlgorithm . 72 5.1.1 WhentoAdjusttheValues . 73 5.1.2 HowtoAdjusttheValues . 74 5.1.3 Example ............................ 76 5.2 AcquiringNewCases. .... ... .... .... .... ... .. 78 5.3 Experiments .............................. 78 5.3.1 SimulationExperiments . 78 5.3.2 RealWorldExperiments . 83 5.4 ConclusionsandFutureWork . 88 6 Experimentation 89 6.1 CBRSystemSettings ......................... 90 6.2 ExperimentsSetup .......................... 95 6.2.1 Robot’sBehaviors. 95 6.2.2 TheScenarios ......................... 96 6.2.3 EvaluationMeasures. 98 6.3 SimulationExperiments . 98 6.3.1 TheSimulator......................... 98 6.3.2 SimulationResults . 99 6.4 RealRobotExperiments . 104 6.4.1 TheRobots........................... 104 6.4.2 Results............................. 106 6.5 ATrialExample............................ 112 6.6 DiscussionandFutureWork. 124 7 Conclusions and Future Work 127 7.1 SummaryoftheContributions . 127 7.2 FutureDirections ... .... ... .... .... .... ... .. 129 Bibliography 133 List of Figures 1.1 Snapshot of the Four-Legged League field . 8 2.1 TheCase-BasedReasoningcycle . 15 3.1 Exampleofaproblemdescription . 35 3.2 Exampleofthescopeofacase. 37 3.3 Exampleofacase ........................... 39 3.4 Exampleofsymmetriccases . 40 3.5 2DGaussianfunction.. .... .... .... ... .... .... 42 3.6 Strategy function combining time and score difference . ..... 43 3.7 Adaptedpositionsexample . 45 3.8 Trapezoidlayoutoftwomatchingpairs . 46 3.9 Correspondenceexamples.. 47 3.10 Ball’strajectoryrepresentation . .. 49 3.11 Example of the ball’s path and the opponents similarity compu- tation.................................. 50 3.12 Sortingexperimentationscenarios . 54 3.13 Trialssortedbyfrequencyofcasesretrieved . ...