Edith Cowan University Research Online Theses : Honours Theses 2007 Using machine learning techniques to create AI controlled players for video games Bhuman Soni Edith Cowan University Follow this and additional works at: https://ro.ecu.edu.au/theses_hons Part of the Artificial Intelligence and Robotics Commons Recommended Citation Soni, B. (2007). Using machine learning techniques to create AI controlled players for video games. https://ro.ecu.edu.au/theses_hons/1294 This Thesis is posted at Research Online. https://ro.ecu.edu.au/theses_hons/1294 Edith Cowan University Copyright Warning You may print or download ONE copy of this document for the purpose of your own research or study. The University does not authorize you to copy, communicate or otherwise make available electronically to any other person any copyright material contained on this site. You are reminded of the following: Copyright owners are entitled to take legal action against persons who infringe their copyright. A reproduction of material that is protected by copyright may be a copyright infringement. Where the reproduction of such material is done without attribution of authorship, with false attribution of authorship or the authorship is treated in a derogatory manner, this may be a breach of the author’s moral rights contained in Part IX of the Copyright Act 1968 (Cth). Courts have the power to impose a wide range of civil and criminal sanctions for infringement of copyright, infringement of moral rights and other offences under the Copyright Act 1968 (Cth). Higher penalties may apply, and higher damages may be awarded, for offences and infringements involving the conversion of material into digital or electronic form. Using Machine Learning Techniques to Create AI Controlled Players for Video Games Dissertation submitted in partial fulfilment of the requirements for the degree of Bachelor of Computer Science (Honours) Honours Thesis: Semester 2, 2007 By: Bhuman Soni Student No: Supervisor: Associate Professor Philip Hingston Faculty: Computing, Health and Science Institution: Edith Cowan University (ECU) Shuman Soni Page 1 22/01/2008 USE OF THESIS The Use of Thesis statement is not included in this version of the thesis. COPYRIGHT AND ACCESS DECLARATION I certify that this thesis does not, to the best of my knowledge and belief: (i) incorporate without acknowledgement any material previously submitted for a degree or diploma in any institution of higher education; (ii) contain any material previously published or written by another person except where due reference is made in the text; or (iii) contain any defamatory material. Signed ..... .' Date ...... 3l.(.O.t/100:6. .................. Abstract This study aims to achieve higher replay and entertainment value in a game through human-like AI behaviour in computer controlled characters called bats. In order to achieve that, an artificial intelligence system capable of learning from observation of human player play was developed. The artificial intelligence system makes use of machine learning capabilities to control the state change mechanism of the bot. The implemented system was tested by an audience of gamers and compared against bats controlled by static scripts. The data collected was focused on qualitative aspects of replay and entertainment value of the game and subjected to quantitative analysis Bhuman Soni Page 2 22/01/2008 Acknowledgements I have to start of by a big thanks to my supervisor Associate Professor Philip Hingston for putting up with me and being incredibly patient through my entire honours journey. Another important person, who's helped me throughout is my Colombian friend Gloria Arenas. Thanks for always being there for me, words cannot express how thankful I am for all that you have done. Also thanks to my parents and sisters whose blessings and wishes have kept my hopes up at all time. Thanks to all my friends and the people who participated in my experiment. I was running my experiment at a busy time of the semester, most of my participants had work commitments and uni commitments and yet they took the time for participation and I really appreciate it. And last but far from the least I thank my god Lord Krishna for I truly believe in him and its because of his blessings I have been able to reach this stage. Shuman Soni Page 3 22/01/2008 Table of contents 1. 1 ......................................................................................................... Introduction 8 2. 2.Background to the study 10 2.1 AI used in games .............................................................................. 10 2.1.1 Artificial neural network ............................................................... 10 2.1.2 Evolutionary algorithms ............................................................... 11 2.1.3 Finite State Machines .................................................................. 12 2.1.4 Fuzzy State Machines (FuSM) ....................................................... 12 2.2 First person shooter genre ................................................................. 13 2.3 Initial Scope of the research ............................................................... 14 3. 3.The significance of the study 16 3.1 Purpose of the Study ......................................................................... 16 3.2 Research questions ............................................. , ............................. 16 4. 4.Review of the Literature 17 4.1 AI and Games ................................................................................... 18 4.2 Machine Learning in Video Games ....................................................... 20 4.3 Research similar to the proposed study ................................................ 22 5. S.Theoretical Framework 25 6. 6 .................................................................................................. Implementation 26 6.1 The AI technique ................................................................................ 26 6.2 The learning algorithm ....................................................................... 27 6.3 The game .........................................................................................30 6.3.1 Some key concepts of Unreal Tournament 2004 .............................. 31 6.4 Testing procedure ............................................................................. 31 6.4.1 Map selection ............................................................................. 32 7. 7.The implemented system 33 7.1 Feedforwa rd Bot ................................................................................ 34 7.2 RecurrentBot .................................................................................... 36 Shuman Soni Page 4 22/01/2008 7.3 UT2004 modifications for the study ..................................................... 38 7.4 The Recorder .................................................................................... 39 7.4.1 The state changes of the data recorder .......................................... 39 7.4.2 Determining the player action ...................................................... .40 7.4.3 Gathering Training data ............................................................... 41 7.5 Training the bots ...............................................................................42 Equalizing the dataset .......................................................................... 42 7.6 The ANN state change mechanism ......................................................43 7.6.1 The ANN bot ...............................................................................44 7.6.2 The game and ANN communication process .................................... 44 8. 8.Implementation phases 47 8.1 Preparation Phase .............................................................................47 8.2 First Implementation Phase ................................................................ 47 8.3 Second Implementation Phase ............................................................ 47 8.4 Third Implementation Phase .............................................................. .4 7 9. 9.The questionnaires 49 9.2 Bot specific questionnaire .................................................................. .49 9.3 Bot rating Questionnaire .................................................................... 49 9.4 Data analysis techniques .................................................................... 50 10. lO.Data analysis: bot specific questionnaire 51 10.1 Unpredictability in bot behavior ......................................................... 52 10.2 Human-like attributes of the bot ....... : ................................................ 54 10.3 Bot's skills and intelligence .............................................................. ,55 10.4 Bot difficulty leve1 ............................................................................ 57 10.5 Bot combat skills ............................................................................. 58 10.6 Entertainment value of the bot .......................................................... 59 10.7 Overall experience playing against the bot .......................................... 61 11. ll.Data Analysis: bot rating questionnaire 63 11.1 Question 1 ...................................................................................... 63 11.2 Question 2 ...................................................................................... 64 11.3 Question 3 .....................................................................................