Planning to Coordinate : Using HTN to Coordinate Unreal® Tournament Bots By Hai Hoang A Thesis Presented to the Graduate and Research Committee of Lehigh University in Candidacy for the Degree of Master of Science in Computer Science and Engineering Lehigh University May of 2005 This thesis is accepted and approved in partial fulfillment of the requirements for the Master of Science. ______________________ Date ______________________ Thesis Advisor ______________________ Chairperson of Department ii Acknowledgements Special thanks to Dr. Hector Muñoz-Avila for his inspiration, guidance, critiques, advice, and unyielding support throughout this project. This project and thesis would have not been realized without him. Thanks to DARPA for sponsoring this research project. Thanks to all my friends at the Intelligent Systems and Technologies Lab (InSyTe). Thank you Stephen Lee-Urban, Marc Ponsen, Ian Warfield, and Ke Xu for all your help and company. And a special thanks to Stephen M. Lee-Urban for all your help with this thesis. iii Table of Contents ABSTRACT. ..................................................................................................................................................1 1. INTRODUCTION.....................................................................................................................................2 2. UNREAL TOURNAMENT......................................................................................................................4 2.1 UNREAL TOURNAMENT BOTS ................................................................................................................6 2.2 GAMEBOT /J AVA BOT .............................................................................................................................7 3. COORDINATING BOTS WITH HTN PLANNING...........................................................................12 3.1 COMPONENTS OF HTN ........................................................................................................................13 3.2 HTN AND UT BOTS .............................................................................................................................14 3.3 HTN AND SHOP .................................................................................................................................16 3.4 STRATEGIES ENCODED WITH HTN S ....................................................................................................17 4. IMPLEMENTATION.............................................................................................................................18 4.1 THE UT SHARED OBJECT ....................................................................................................................20 4.2. THE UT SERVER .................................................................................................................................20 4.3 THE JAVABOT ......................................................................................................................................21 4.3.1 CMU_Jbot’s Bugs and Solutions................................................................................................21 4.3.2 CMU_Jbot’s Dominating, Navigating, and Exploring Algorithms.............................................24 4.3.3 HTNbot: Combining CMU_Jbot and HTN Planning..................................................................24 4.4 SHOP..................................................................................................................................................25 4.5 AN OVERVIEW OF THE PROCESS ..........................................................................................................26 5. INSTALLATION ....................................................................................................................................27 5.1 INSTALLING THE UT SERVER ...............................................................................................................27 5.2 INSTALLING THE GAMEBOT .................................................................................................................28 5.3 INSTALLING THE TCL VIZ .....................................................................................................................29 5.4 INSTALLING THE SOURCE CODE FOR THE JAVABOT .............................................................................30 6. EXPERIMENTS......................................................................................................................................31 6.1 EXPERIMENTS FOR THE AIIDE-05 PAPER ............................................................................................31 6.2 THE SECOND SET OF EXPERIMENTS .....................................................................................................33 7. LIMITATIONS .......................................................................................................................................35 8. FUTURE WORK ....................................................................................................................................36 9. CONCLUSION........................................................................................................................................38 10. REFERENCE ........................................................................................................................................39 11. VITA.......................................................................................................................................................40 iv List of Figures Figure 1. A Bot Dominating a Dompoint. ........................................................................ 5 Figure 2. The Gamebot Architecture ................................................................................ 9 Figure 3. The Javabot Architechture............................................................................... 10 Figure 4. BotRunnerApp’s User Interface ...................................................................... 12 Figure 5. A Method in HTN for UT Bots ....................................................................... 14 Figure 6. Dataflow of HTN Planning for Coordinating UT Bots ................................... 15 Figure 7. A Bot Following the Pathmarkers. .................................................................. 28 Figure 8. TclViz – A visualization tool for UT............................................................... 30 Figure 9. Result for Control Half Plus One vs. Standard................................................ 32 Figure 10. Result for Control Half Plus One vs. Improved ............................................ 33 Figure 11. Result for Control All Points vs. Standard ................................................... 33 Figure 12. Result for Control All Points vs. Improved................................................... 33 Figure 13. Results of the Dynamic Domination Strategy ............................................... 34 v Abstract. Using Hierarchical Task Network (HTN) planning representations to encode strategic game AI in a first-person-shooter (FPS) game has been proposed by (Munoz- Avila & Fisher, 2004) but this was preliminary work and no specific application or implementation of HTN in games were provided. This thesis presents a research project, which was funded by the Defense Advanced Research Projects Agency (DARPA), showing that it is possible to use HTN to model team-based strategies in a FPS game by providing an actual implementation. The HTN planner used to create the HTN plans is SHOP (Nau et al ., 1999) and the client used to control the bots in UT is Javabot (Javabot, 2005). It also presents the results of the experiments of this implementation where HTN representations are used to encode strategies that coordinate a team of bots in an Unreal Tournament© (UT) Domination game. The experiment consists of two simple strategies encoded in HTN and a third that is a combination of the two and the bot dynamically selects which of the two strategies is more applicable to execute. A paper describing the two simple strategies and the results of the experiments has been accepted by the First Annual Artificial Intelligence for Interactive Digital Entertainment Conference (AIIDE- 05). The results from that paper will also be presented in this thesis along with the result for the third strategy. The results are very promising and they should stimulate further research that utilizes HTN in game AI. 1 1. Introduction Unreal Tournament is a multiplayer, First-Person Shooter (FPS) game developed by Epic Games, Inc. It uses a client/server architecture that allows different client programs to control the behaviors of bots, and one such client is the Javabot which could control multiple bots at the same time. Currently, Javabot uses a finite state machine (FSM) to control the individual behaviors of a bot. Each bot is controlled individually so these bots could not effectively communicate with each other even in a team based game. The AI for the bots that came with UT is very good with fragging games such as Deathmatch because the bot’s ability to navigate, run, and shoot is very good. A frag, usually associated with multiplayer Deathmatch games, refers to a kill of an opponent. GameArea.com said that “their basic death-matching skills have been virtually perfected; their understanding of all the different game rules is perfect; their ability to fully navigate levels